System and method for providing interactive advertisement

ABSTRACT

In example embodiments, a system and method for providing interactive advertisement is provided. In some embodiments, a component causes a video program having an interactive advertisement to be displayed to a user on a display screen. The interactive advertisement includes an enhanced content activator. In response to activation of the enhanced content activator by the user, enhanced advertisement information including at least one trigger is provisioned. The enhanced advertisement information is then caused to be displayed with the video program and the interactive advertisement. A selection by the user of the at least one trigger associated with the enhanced advertisement information is monitored. In response to selection of the at least one trigger, access to further content associated with the enhanced content information is provided.

RELATED APPLICATION

The present application is a continuation-in-part U.S. patentapplication Ser. No. 13/175,663, filed Jul. 1, 2011 and entitled“Advertising Methods for Advertising Time Slots and Embedded Objects,”which is a continuation of U.S. Pat. No. 7,979,877, issued Jul. 12, 2011and entitled “Advertising Methods for Advertising Time Slots andEmbedded Objects,” both of which applications are hereby incorporated byreference herein in their entirety.

FIELD

The present disclosure relates generally to advertising, and in aspecific example embodiment, to providing interactive advertisement.

BACKGROUND

Typically, advertisements are presented during breaks between videoprogramming. These commercial breaks generally are not interactive andtypically fail to provide additional information than what is shown onthe advertisement. As a result, a viewer may be forced to independentlyresearch for the additional information.

BRIEF DESCRIPTION OF DRAWINGS

The appended drawings merely illustrate example embodiments of thepresent invention and cannot be considered as limiting its scope.

FIG. 1 is a schematic illustration of the overall system of the presentinvention

FIG. 2 is a detailed block diagram of a set-top box, display, and remotecontrol device of the system of the present invention.

FIG. 3 is an illustration of an embodiment of the present inventionwherein four video signals may be combined into four composite videosignals.

FIG. 4 is an illustration of an embodiment of the present inventionwherein a main video image is combined with portions of a second videoimage to create five composite video signals.

FIG. 5 depicts another set top box embodiment of the present invention.

FIG. 6 depicts a sequence of steps employed to create a combined imageat a user's set top box.

FIG. 7 is an illustration of one implementation of the presentinvention.

FIG. 8 is an illustration of the manner in which interactive content maybe displayed.

FIG. 9 is a graphical representation of a database that provides a listof additional information associated with advertisers.

FIG. 10 is a graphical representation of a database that provides a listof attributes associated with advertisers.

FIGS. 11 and 12 are illustrations of scenes showing otherimplementations of the present invention.

FIGS. 13, 14, and 15 are illustrations of scenes showing otherimplementations of the present invention.

FIGS. 16 and 17 are illustrations of scenes showing otherimplementations of the present invention.

FIG. 18 is a schematic block diagram that generally illustrates oneembodiment of the present invention.

FIG. 19 is a schematic block diagram that generally illustrates anotherembodiment of the present invention.

FIG. 20 is a flow diagram that illustrates the steps that may beperformed in applying embedded ads into a video stream.

FIG. 21 is a flow diagram illustrating the steps that may be performedfor the installation of embedded ads by manual designation of availableadvertising space.

FIG. 22 is a flow diagram illustrating the steps that may be performedfor the installation of embedded ads by automatic designation ofavailable advertising space and searching for actual objects.

FIG. 23 is a flow diagram illustrating the steps that may be performedfor the installation of embedded ads by automatic designation ofavailable advertising space and searching for blank spaces.

FIG. 24 is a flow diagram illustrating the steps that may be performedfor the installation of embedded ads, using automatic methods, in amanner which combines the processes of looking for an actual object andblank space simultaneously.

FIG. 25 is a flow diagram illustrating the steps that may be performedfor the installation of embedded ads by using tags and markers todescribe advertising space.

FIG. 26 is a flow diagram illustrating a process of auctioning embeddedadvertising.

FIG. 27 is a flow diagram illustrating a process of biding on embeddedadvertising.

FIG. 28 is a flow diagram illustrating a process of modifying biddingstrategy based upon affinity data.

FIG. 29 is a flow diagram illustrating a process of bidding on liveevents

FIG. 30 is a flow diagram illustrating a process of analyzingadvertising opportunities for live events.

FIG. 31 is a flow diagram illustrating a process for billing advertisersbased upon display time of embedded ads.

FIG. 32a is an alternative embodiment of FIG. 7 providing access tosecondary content.

FIG. 32b is an alternative embodiment of FIG. 8 providing access tosecondary content.

FIG. 33 is a block diagram illustrating an example embodiment of amanagement system.

FIG. 34 is a flow diagram illustrating an example high-level method forproviding further content based on an interactive advertisement.

FIG. 35 is a flow diagram illustrating a more detailed method forproviding the further content based on the interactive advertisement.

FIG. 36 is an action flow diagram for providing further content based onan interactive advertisement.

FIG. 37 is a simplified block diagram of a machine in an example form ofa computing system within which a set of instructions for causing themachine to perform any one or more of the methodologies discussed hereinmay be executed.

DETAILED DESCRIPTION

The description that follows includes systems, methods, techniques,instruction sequences, and computing machine program products thatembody illustrative embodiments of the present invention. In thefollowing description, for purposes of explanation, numerous specificdetails are set forth in order to provide an understanding of variousembodiments. It will be evident, however, to those skilled in the artthat embodiments may be practiced without these specific details. Ingeneral, well-known instruction instances, protocols, structures, andtechniques have not been shown in detail.

Example embodiments described herein provide systems and methods forproviding interactive advertisement. In some embodiments, theinteractive advertisement is embedded within a portion of a videoprogram such as a scene of the video program. An activation indicationis received from a user device displaying the video program having theinteractive advertisement that is selectable by a user to access furthercontent. The activation indication triggers provisioning of enhancedadvertisement information including selectable triggers to access thefurther content associated with the interactive advertisement. Theenhanced advertisement information may be provided in an unused or lowactivity portion of the video program. Alternatively, the enhancedadvertisement information may be provided in an appended portionbordering a scene of the video program.

The enhanced advertisement information including the selectable triggersare provided to the user device to be displayed with the interactiveadvertisement. A selection of the selectable trigger is then received.The selection provides access to the further content associated with theinteractive advertisement. In some embodiments, the further contentcomprises secondary content that is associated with the interactiveadvertisement but is not a primary product being advertised in theinteractive advertisement. The further content may be accessed by, forexample, providing digital content for download, providing access toinformation from the Internet, or providing an order interface topurchase a product.

In some embodiments, an advertisement opportunity may be identifiedwhere the interactive advertisement is to be placed. This may be done bydetecting an object in the video program where the interactiveadvertisement is to be placed, whereby the detecting comprises using apattern recognition process to detect the object and to generateplacement and contour data for the detected object.

FIGS. 1 through 6 illustrate one method of generating embeddedadvertising using video combination techniques. The combination ofsignals may occur in the set top box or may occur in an upstream source(e.g., national broadcaster, local station). The embedded advertisingcan take the form of objects that are inserted into a video (e.g.,television) programming or other digital media using overlay pages thatare generated by markup language, by video combination techniques, or byany other desirable technique. Embedded advertising using overlay pagescan replace objects in, for example, the television broadcast bynegating the television broadcast display on a pixel by pixel basis sothat objects can actually be eliminated from the broadcast, such asdisclosed in U.S. patent application Ser. No. 10/212,289, entitled “PostProduction Visual Alterations,” by Thomas Lemmons, filed on Aug. 2,2002, which is specifically incorporated herein by reference for allthat it discloses and teaches. Alternatively, video combinationtechniques can be used to embed ads by combining video images at the settop box level, such as disclosed in U.S. patent application Ser. No.10/103,545, entitled “Video Combiner,” Thomas Lemmons, et al., filedMar. 20, 2002, which is specifically incorporated herein by referencefor all that it discloses and teaches. Inserted objects can track withthe background of the display and conform to objects in the display sothat the inserted objects appear as objects that are actually containedwithin the television broadcast such as disclosed in U.S. patentapplication Ser. No. 10/357,155, entitled “Tracking Moving Objects onVideo with Interactive Access Points”, filed Jan. 31, 2003, by Steven O.Markel which is specifically incorporated herein by reference for allthat it discloses and teaches. Embedded advertising provides a highdegree of flexibility for offering (e.g., auctioning or selling) ads.This high degree of flexibility allows television broadcasters to offerembedded advertising which can be bid upon by television advertisers forboth prerecorded and live programming.

FIG. 1 illustrates the interconnections of the various components thatmay be used to deliver a composite video signal to individual viewers.Video sources 100 and 126 send video signals 102 and 126 through adistribution network 104 to viewer's locations 111. Additionally,multiple interactive video servers 106 and 116 send video, HTML, andother attachments 108. The multiple feeds 110 are sent to several settop boxes 112, 118, and 122 connected to televisions 114, 120, and 124,respectively. The set top boxes 112 and 118 may be interactive set topboxes and set top box 122 may not have interactive features.

The video sources 100 and 126 and interactive video servers 106 and 116may be attached to a conventional cable television head-end, a satellitedistribution center, or other centralized distribution point for videosignals. The distribution network 104 may comprise a cable televisionnetwork, satellite television network, Internet video distributionnetwork, or any other network capable of distributing video ormultimedia data.

The interactive set top boxes 112 and 118 may communicate to theinteractive video servers 106 and 108 though the video distributionnetwork 104 if the video distribution network supports two-waycommunication, such as with cable modems. Additionally, communicationmay be through other upstream communication networks 130. Such upstreamnetworks may include a dial up modem, direct Internet connection, orother communication network that allows communication separate from thevideo distribution network 104.

Although FIG. 1 illustrates the use of interactive set-top boxes 112 and118, the present invention can be implemented without an interactiveconnection with an interactive video server, such as interactive videoservers 106 and 116. In that case, separate multiple video sources 100can provide multiple video feeds 110 to non-interactive set-top box 122at the viewer's locations 111. The difference between the interactiveset top boxes 112 and 118 and the non-interactive set top box 122 isthat the interactive set top boxes 112 and 118 incorporate thefunctionality to receive, format, and display interactive content andsend interactive requests to the interactive video servers 106 and 116.

The set top boxes 112, 118, and 122 may receive and decode two or morevideo feeds and combine the feeds to produce a composite video signalthat is displayed for the viewer. Such a composite video signal may bedifferent for each viewer, since the video signals may be combined inseveral different manners. The manner in which the signals are combinedis described in the presentation description. The presentationdescription may be provided through the interactive video servers 106and 116 or through another server 132. Server 132 may be a web server ora specialized data server.

As disclosed below, the set-top box includes multiple video decoders anda video controller that provides control signals for combining the videosignal that is displayed on the display 114. In accordance withcurrently available technology, the interactive set-top box 112 canprovide requests to the interactive video server 106 to provide variousweb connections for display on the display 114. Multiple interactivevideo servers 116 can provide multiple signals to the viewer's locations111.

The set top boxes 112, 118, and 122 may be a separate box thatphysically rests on top of a viewer's television set, may beincorporated into the television electronics, may be functions performedby a programmable computer, or may take on any other form. As such, aset top box refers to any receiving apparatus capable of receiving videosignals and employing a presentation description as disclosed herein.

The manner in which the video signals are to be combined is defined inthe presentation description. The presentation description may be aseparate file provided by the server 132, the interactive video servers106 and 116, or may be embedded into one or more of the multiple feeds110. A plurality of presentation descriptions may be transmitted andprogram code operating in a set top box may select one or more of thepresentation descriptions based upon an identifier in the presentationdescription(s). This allows presentation descriptions to be selectedthat correspond to set top box requirements and/or viewer preferences orother information. Further, demographic information may be employed byupstream equipment to determine a presentation description version for aspecific set top box or group of set top boxes and an identifier of thepresentation description version(s) may then be sent to the set top boxor boxes. Presentation descriptions may also be accessed across anetwork, such as the Internet, that may employ upstream communication ona cable system or other networks. In a similar manner, a set top box mayaccess a presentation description across a network that corresponds toset top box requirements and/or viewer preferences or other information.And in a similar manner as described above, demographic information maybe employed by upstream equipment to determine a presentationdescription version for a specific set top box or group of set top boxesand an identifier of the presentation description version(s) may then besent to the set top box or boxes. The identifier may comprise a URL,filename, extension or other information that identifies thepresentation description. Further, a plurality of presentationdescriptions may be transferred to a set top box and a viewer may selectversions of the presentation description. Alternatively, softwareprogram operating in the set top box may generate the presentationdescription and such generation may also employ viewer preferences ordemographic information.

In some cases, the presentation description may be provided by theviewer directly into the set top box 112, 118, 122, or may be modifiedby the viewer. Such a presentation description may be viewer preferencesstored in the set top box and created using menus, buttons on a remote,a graphical viewer interface, or any combination of the above. Othermethods of creating a local presentation description may also be used.

The presentation description may take the form of a markup languagewherein the format, look, and feel of a video image is controlled. Usingsuch a language, the manner in which two or more video images arecombined may be fully defined. The language may be similar to XML, HTMLor other graphical mark-up languages and allow certain video functionssuch as pixel by pixel replacement, rotation, translation, and deformingof portions of video images, the creation of text and other graphicalelements, overlaying and ghosting of one video image with another, colorkey replacement of one video image with another, and any other commandas may be contemplated. In contrast to hard-coded image placementchoices typical to picture-in-picture (PIP) display, the presentationdescription of the present invention is a “soft” description thatprovides freedom in the manner in which images are combined and that maybe easily created, changed, modified or updated. The presentation is notlimited to any specific format and may employ private or public formatsor a combination thereof. Further, the presentation description maycomprise a sequence of operations to be performed over a period of timeor over a number of frames. In other words, the presentation descriptionmay be dynamic. For example, a video image that is combined with anothervideo image may move across the screen, fade in or out, may be alteredin perspective from frame to frame, or may change in size.

Specific presentation descriptions may be created for each set top boxand tailored to each viewer. A general presentation description suitedto a plurality of set top boxes may be parsed, translated, interpreted,or otherwise altered to conform to the requirements of a specific settop box and/or to be tailored to correspond to a viewer demographic,preference, or other information. For example, advertisements may betargeted at selected groups of viewers or a viewer may have preferencesfor certain look and feel of a television program. In some instances,some presentation descriptions may be applied to large groups ofviewers.

The presentation descriptions may be transmitted from a server 132 toeach set top box through a backchannel 130 or other network connection,or may be embedded into one or more of the video signals sent to the settop box. Further, the presentation descriptions may be sent individuallyto each set top box based on the address of the specific set top box.Alternatively, a plurality of presentation descriptions may betransmitted and a set top box may select and store one of thepresentation descriptions based upon an identifier or other informationcontained in the presentation description. In some instances, the settop box may request a presentation description through the backchannel130 or through the video distribution network 104. At that point, aserver 132, interactive video server 106 or 116, or other source for apresentation description may send the requested presentation descriptionto the set top box.

Interactive content supplied by interactive video server 106 or 116 mayinclude the instructions for a set top box to request the presentationdescription from a server through a backchannel. A methodology fortransmitting and receiving this data is described in U.S. ProvisionalPatent Application entitled “Multicasting of Interactive Data Over aBack Channel”, filed Mar. 5, 2002 by Ian Zenoni, which is specificallyincorporated herein by reference for all it discloses and teaches.

The presentation description may contain the commands necessary forseveral combinations of video. In such a case, the local preferences ofthe viewer, stored in the set top box, may indicate which set ofcommands would be used to display the specific combination of videosuitable for that viewer. For example, in an advertisement campaign, apresentation description may include commands for combining severalvideo images for four different commercials for four different products.The viewer's preferences located inside the set top box may indicate apreference for the first commercial, thusly the commands required tocombine the video signals to produce the first commercial will beexecuted and the other three sets of commands will be ignored.

In operation, the device of FIG. 1 provides multiple video feeds 110 tothe viewer's locations 111. The multiple video feeds are combined byeach of the interactive set-top boxes 112, 118, 122 to generatecorrelated or composite video signals 115, 117, 119, respectively. Asdisclosed below, each of the interactive set-top boxes 112, 118, 122uses instructions provided by the video source 100, interactive videoservers 106, 116, a separate server 132, or viewer preferences stored atthe viewer's location to generate control signals to combine the signalsinto a correlated video signal. Additionally, presentation descriptioninformation provided by each of the interactive video servers 106, 116can provide layout descriptions for displaying a video attachment. Thecorrelated video signal may overlay the various video feeds on a fullscreen basis, or on portions of the screen display. In any event, thevarious video feeds may interrelate to each other in some fashion suchthat the displayed signal is a correlated video signal with interrelatedparts provided by each of the separate video feeds.

FIG. 2 is a detailed schematic block diagram of an interactive set-topbox together with a display 202 and remote control device 204. As shownin FIG. 2, a multiple video feed signal 206 is supplied to theinteractive set-top box 200. The multiple video feed signal 206 thatincludes a video signal, HTML signals, video attachments, a presentationdescription, and other information is applied to a tuner/decoder 208.The tuner/decoder 208 extracts each of the different signals such as avideo MPEG signal 210, an interactive video feed 212, another video orinteractive video feed 214, and the presentation description information216.

The presentation description information 216 is the informationnecessary for the video combiner 232 to combine the various portions ofmultiple video signals to form a composite video image. The presentationdescription information 216 can take many forms, such as an ATVEFtrigger or a markup language description using HTML or a similar format.Such information may be transmitted in a vertical blanking encodedsignal that includes instructions as to the manner in which to combinethe various video signals. For example, the presentation description maybe encoded in the vertical blanking interval (VBI) of stream 210. Thepresentation description may also include Internet addresses forconnecting to enhanced video web sites. The presentation descriptioninformation 216 may include specialized commands applicable tospecialized set top boxes, or may contain generic commands that areapplicable to a wide range of set top boxes. References made herein tothe ATVEF specification are made for illustrative purposes only, andsuch references should not be construed as an endorsement, in anymanner, of the ATVEF specification.

The presentation description information 216 may be a program that isembedded into one or more of the video signals in the multiple feed 206.In some cases, the presentation description information 216 may be sentto the set top box in a separate channel or communication format that isunrelated to the video signals being used to form the composite videoimage. For example, the presentation description information 216 maycome through a direct internet connection made through a cable modem, adial up internet access, a specialized data channel carried in themultiple feed 206, or any other communication method.

As also shown in FIG. 2, the video signal 210 is applied to a videodecoder 220 to decode the video signal and apply the digital videosignal to video RAM 222 for temporary storage. The video signal 210 maybe in the MPEG standard, wherein predictive and intra-coded framescomprise the video signal. Other video standards may be used for thestorage and transmission of the video signal 210 while maintainingwithin the spirit and intent of the present invention. Similarly, videodecoder 224 receives the interactive video feed 212 that may comprise avideo attachment from an interactive web page. The video decoder 224decodes the video signal and applies it to a video RAM 226. Videodecoder 228 is connected to video RAM 230 and operates in the samefashion. The video decoders 220, 224, 228 may also perform decompressionfunctions to decompress MPEG or other compressed video signals. Each ofthe video signals from video RAMs 222, 226, 230 is applied to a videocombiner 232. Video combiner 232 may comprise a multiplexer or otherdevice for combining the video signals. The video combiner 232 operatesunder the control of control signals 234 that are generated by the videocontroller 218. In some embodiments of the present invention, ahigh-speed video decoder may process more than one video feed and thefunctions depicted for video decoders 220, 224, 228 and RAMs 222, 226,230 may be implemented in fewer components. Video combiner 232 mayinclude arithmetic and logical processing functions.

The video controller 218 receives the presentation descriptioninstructions 216 and generates the control signals 234 to control thevideo combiner 232. The control signals may include many commands tomerge one video image with another. Such commands may include directoverlay of one image with another, pixel by pixel replacement, colorkeyed replacement, the translation, rotation, or other movement of asection of video, ghosting of one image over another, or any othermanipulation of one image and combination with another as one mightdesire. For example, the presentation description instructions 216 mayindicate that the video signal 210 be displayed on full screen while theinteractive video feed 212 only be displayed on the top third portion ofthe screen.

The presentation description instructions 216 also instruct the videocontroller 218 as to how to display the pixel information. For example,the control signals 234 generated by the video controller 218 mayreplace the background video pixels of video 210 in the areas where theinteractive video feed 212 is applied on the top portion of the display.The presentation description instructions 216 may set limits as toreplacement of pixels based on color, intensity, or other factors.Pixels can also be displayed based upon the combined output of each ofthe video signals at any particular pixel location to provide a trulycombined output signal. Of course, any desired type of combination ofthe video signals can be obtained, as desired, to produce the combinedvideo signal 236 at the output of the video combiner 232. Also, anynumber of video signals can be combined by the video combiner 232 asillustrated in FIG. 2. It is only necessary that a presentationdescription 216 be provided so that the video controller 218 cangenerate the control signals 234 that instruct the video combiner 232 toproperly combine the various video signals.

The presentation description instructions 216 may include instructionssent from a server directly to the set top box 200 or the presentationdescription instructions 216 may be settable by the viewer. For example,if an advertisement were to be shown to a specific geographical area,such as to the viewers in a certain zip code, a set of presentationdescription instructions 216 may be embedded into the advertisementvideo instructing the set top box 200 to combine the video in a certainmanner or in a certain location.

In some embodiments, the viewer's preferences may be stored in the localpreferences 252 and used either alone or in conjunction with thepresentation description instructions 216. For example, the localpreferences may be to merge a certain preferred background with a newsshow. In another example, the viewer's local preferences may select froma list of several options presented in the presentation descriptioninformation 216. In such an example, the presentation descriptioninformation 216 may contain the instructions for several alternativepresentation schemes, one of which may be preferred by a viewer andcontained in the local preferences 252.

In some embodiments, the viewer's preferences may be stored in a centralserver. Such an embodiment may provide for the collection and analysisof statistics regarding viewer preferences. Further, customized andtargeted advertisements and programming preferences may be sent directlyto the viewer, based on their preferences analyzed on a central server.The server may have the capacity to download presentation descriptioninstructions 216 directly to the viewer's set top box. Such a downloadmay be pushed, wherein the server sends the presentation descriptioninstructions 216, or pulled, wherein the set top box requests thepresentation description instructions 216 from the server.

As also shown in FIG. 2, the combined video signal 236 is applied to aprimary rendering engine 238. The primary rendering engine 238 generatesthe correlated video signal 240. The primary rendering engine 238formats the digital combined video signal 236 to produce the correlatedvideo signal 240. If the display 202 is an analog display, the primaryrendering engine 238 also performs functions as a digital-to-analogconverter. If the display 202 is a high definition digital display, theprimary rendering engine 238 places the bits in the proper format in thecorrelated video signal 240 for display on the digital display.

FIG. 2 also discloses a remote control device 204 under the operation ofa viewer. The remote control device 204 operates in the standard fashionin which remote control devices interact with interactive set-top boxes,such as interactive set-top box 200. The set-top box includes a receiver242 such as an infrared (IR) receiver that receives the signal 241 fromthe remote 204. The receiver 242 transforms the IR signal into anelectrical signal that is applied to an encoder 244. The encoder 244encodes the signal into the proper format for transmission as aninteractive signal over the digital video distribution network 104. Thesignal is modulated by modulator 246 and up-converted by up-converter248 to the proper frequency. The up-converted signal is then applied toa directional coupler 250 for transmission on the multiple feed 206 tothe digital video distribution network 104. Other methods of interactingwith an interactive set top box may be also employed. For example,viewer input may come through a keyboard, mouse, joystick, or otherpointing or selecting device. Further, other forms of input, includingaudio and video may be used. The example of the remote control 204 isexemplary and not intended to limit the invention.

As also shown in FIG. 2, the tuner/decoder 208 may detect web addressinformation 215 that may be encoded in the video signal 102 (FIG. 1).This web address information may contain information as to one or moreweb sites that contain presentation descriptions that interrelates tothe video signal 102 and that can be used to provide the correlatedvideo signal 240. The decoder 208 detects the address information 215which may be encoded in any one of several different ways such as anATVEF trigger, as a tag in the vertical blanking interval (VBI), encodedin the back channel, embedded as a data PID (packet identifier) signalin a MPEG stream, or other encoding and transmitting method. Theinformation can also be encoded in streaming media in accordance withMicrosoft's ASF format. Encoding this information as an indicator ismore fully disclosed in U.S. patent application Ser. No. 10/076,950,filed Feb. 12, 2002 entitled “Video Tags and Markers,” which isspecifically incorporated herein by reference for all that it disclosesand teaches. The manner in which the tuner/decoder 208 can extract theone or more web addresses 215 is more fully disclosed in the abovereferenced patent application. In any event, the address information 215is applied to the encoder 244 and is encoded for transmission throughthe digital video distribution network 104 to an interactive videoserver. The signal is modulated by modulator 246 and up-converted byup-converter 248 for transmission to the directional coupler 250 overthe cable. In this fashion, video feeds can automatically be provided bythe video source 100 via the video signal 102.

The web address information that is provided can be selected, asreferenced above, by the viewer activating the remote control device204. The remote control device 204 can comprise a personalized remote,such as disclosed in U.S. patent Ser. No. 09/941,148, filed Aug. 27,2001 entitled “Personalized Remote Control,” which is specificallyincorporated by reference for all that it discloses and teaches.Additionally, interactivity using the remote 204 can be provided inaccordance with U.S. patent application Ser. No. 10/041,881, filed Oct.24, 2001 entitled “Creating On-Content Enhancements,” which isspecifically incorporated herein by reference for all that it disclosesand teaches. In other words, the remote 204 can be used to access “hotspots” on any one of the interactive video feeds to provide furtherinteractivity, such as the ability to order products and services, andother uses of the “hot spots” as disclosed in the above referencedpatent application. Preference data can also be provided in an automatedfashion based upon viewer preferences that have been learned by thesystem or are selected in a manual fashion using the remote controldevice in accordance with U.S. patent application Ser. No. 09/933,928,filed Aug. 21, 2001, entitled “iSelect Video” and U.S. patentapplication Ser. No. 10/080,996, filed Feb. 20, 2002 entitled “ContentBased Video Selection,” both of which are specifically incorporated byreference for all that they disclose and teach. In this fashion,automated or manually selected preferences can be provided to generatethe correlated video signal 240.

FIG. 3 illustrates an embodiment 300 of the present invention whereinfour video signals, 302, 304, 306, and 308, may be combined into fourcomposite video signals 310, 312, 314, and 316. The video signals 302and 304 represent advertisements for two different vehicles. Videosignal 302 shows an advertisement for a sedan model car, where videosignal 304 shows an advertisement for a minivan. The video signals 306and 308 are background images, where video signal 306 shows a backgroundfor a mountain scene and video signal 308 shows a background for anocean scene. The combination or composite of video signals 306 and 302yields signal 310, showing the sedan in front of a mountain scene.Similarly, the signals 312, 314, and 316 are composite video signals.

In the present embodiment, the selection of which composite image todisplay on a viewer's television may be made in part with a localpreference for the viewer and by the advertiser. For example, theadvertiser may wish to show a mountain scene to those viewers fortunateenough to live in the mountain states. The local preferences may dictatewhich car advertisement is selected. In the example, the localpreferences may determine that the viewer is an elderly couple with nochildren at home and thus may prefer to see an advertisement for a sedanrather than a minivan.

The methodology for combining the various video streams in the presentembodiment may be color key replacement. Color key replacement is amethod of selecting pixels that have a specific color and location andreplacing those pixels with the pixels of the same location from anothervideo image. Color key replacement is a common technique used in theindustry for merging two video images.

FIG. 4 illustrates an embodiment 400 of the present invention wherein amain video image 402 is combined with portions of a second video image404. The second video image 404 comprises four small video images 406,408, 410, and 412. The small images may be inserted into the main videoimage 402 to produce several composite video images 414, 416, 418, 420,and 422.

In the embodiment 400, the main video image 402 comprises a border 424and a center advertisement 426. In this case, the border describestoday's special for Tom's Market. The special is the centeradvertisement 426, which is shrimp. Other special items are shown in thesecond video image 404, such as fish 406, ham 408, soda 410, and steak412. The viewer preferences may dictate which composite video is shownto a specific viewer. For example, if the viewer is a vegetarian,neither the ham 408 nor steak 412 advertisements would be appropriate.If the person had a religious preference that indicated that they wouldeat fish on a specific day of the week, for example, the fish special406 may be offered. If the viewer's preferences indicated that theviewer had purchased soda from the advertised store in the past, thesoda advertisement 410 may be shown. In cases where no preference isshown, a random selection may be made by the set top box, a defaultadvertisement, or other method for selecting an advertisement may beused. While FIG. 4 provides the center advertisement 426, it is notedthat the advertisement may be located anywhere on the main video image402.

Hence, the present invention provides a system in which a correlated orcomposite video signal can be generated at the viewer location. Anadvantage of such a system is that multiple video feeds can be providedand combined as desired at the viewer's location. This eliminates theneed for generating separate combined video signals at a productionlevel and transmission of those separate combined video signals over atransmission link. For example, if ten separate video feeds are providedover the transmission link, a total of ten factorial combined signalscan be generated at the viewer's locations. This greatly reduces thenumber of signals that have to be transmitted over the transmissionlink.

Further, the present invention provides for interactivity in both anautomated, semi-automated, and manual manner by providing interactivevideo feeds to the viewer location. As such, greater flexibility can beprovided for generating a correlated video signal.

FIG. 5 depicts another set top box embodiment of the present invention.Set top box 500 comprises tuner/decoder 502, decoder 504, memory 506,processor 508, optional network interface 510, video output unit 512,and user interface 514. Tuner/decoder 502 receives a broadcast thatcomprises at least two video signals 501. In one embodiment of FIG. 5,the tuner/decoder 502 is capable of tuning at least two independentfrequencies. In another embodiment of FIG. 5, the tuner/decoder 502decodes at least two video signals contained within a broadcast band, asmay occur with QAM or QPSK transmission over analog television channelbands or satellite bands. “Tuning” of video signals may compriseidentifying packets with predetermined PID (Packet Identifiers) valuesor a range thereof and forwarding such packets to processor 508 or todecoder 504. For example, audio and/or video data packets 503 may betransferred to decoder 504 and control packets may be transferred toprocessor 508. Data packets may be discerned from control packets 507through secondary PIDs or through PID values in a predetermined range.Decoder 504 processes packets received from tuner/decoder 502 andgenerates and stores image and/or audio information 505 in memory 506.Image and audio information 505 may comprise various information typescommon to DCT based image compression methods, such as MPEG and motionJPEG, for example, or common to other compression methods such aswavelets and the like. Audio information may conform to MPEG or otherformats such as those developed by Dolby Laboratories and THX as arecommon to theaters and home entertainment systems. Decoder 504 maycomprise one or more decoder chips to provide sufficient processingcapability to process two or more video streams substantiallysimultaneously. Control packets 507 provided to processor 508 mayinclude presentation description information. Presentation descriptioninformation may also be accessed employing network interface 510.Network interface 510 may be connected to any type of network 511 thatprovides access to a presentation description including modems, cablemodems, DSL modems, upstream channels in a set top box and the like.Network interface 510 may also be employed to provide user responses tointeractive content to an associated server or other equipment.

Processor 508 of FIG. 5 employs the presentation description to controlthe combination of the image and/or audio information stored in memory506. The combining process may employ processor 508, decoder 504, or acombination of processor 508 and decoder 504. The combined image and oraudio information, as created employing the presentation description, issupplied to video output unit 512 that produces and output signal for atelevision, monitor, or other type of display 515. The output signal 516may comprise composite video, S-video, RGB, or any other format. Userinterface 514 supports a remote control, mouse, keyboard or other inputdevice. User input 517 may serve to select versions of a presentationdescription or to modify a presentation description via user interface514.

FIG. 6 depicts a sequence of steps 600 employed to create a combinedimage at a user's set top box. At step 602 a plurality of video signalsare received. These signals may contain digitally encoded image andaudio data. At step 604 a presentation description is accessed. Thepresentation description may be part of a broadcast signal or may beaccessed across a network. At step 606, at least two of the videosignals are decoded and image data and audio data (if present) for eachvideo signal is stored in a memory of the set top box. At step 608,portions of the video images and optionally portions of the audio dataare combined in accordance with the presentation description. Thecombination of video images and optionally audio data may producecombined data in the memory of the set top box, or such combination maybe performed “on the fly” wherein real-time combination is performed andthe output provided to step 610. For example, if a mask is employed toselect between portions of two images, non-sequential addressing of theset top box memory may be employed to access portions of each image in areal-time manner, eliminating the need to create a final display imagein set top box memory. At step 610 the combined image and optionallycombined audio are output to a presentation device such as a television,monitor, or other display device. Audio may be provided to thepresentation device or to an amplifier, stereo system, or other audioequipment.

The presentation description of the present invention provides adescription through which the method and manner in which images and/oraudio streams are combined may be easily be defined and controlled. Thepresentation description may specify the images to be combined, thescene locations at which images are combined, the type of operation oroperations to be performed to combine the images, and the start andduration of display of combined images. Further, the presentationdescription may include dynamic variables that control aspects ofdisplay such as movement, gradually changing perspective, and similartemporal or frame varying processes that provide image modification thatcorresponds to changes in scenes to which the image is applied.

Images to be combined may be processed prior to transmission or may beprocessed at a set top box prior to display or both. For example, animage that is combined with a scene as the scene is panned may beclipped to render the portion corresponding to the displayed image suchthat a single image may be employed for a plurality of video frames.

The combination of video images may comprise the process of replacingand/or combining a portion of a first video image with a second videoimage. The manner in which images are combined may employ any hardwareor software methods and may include bit-BLTs (bit block logictransfers), raster-ops, and any other logical or mathematical operationsincluding but not limited to maxima, minima, averages, gradients, andthe like. Such methods may also include determining an intensity orcolor of an area of a first image and applying the intensity or color toan area of a second image. A color or set of colors may be used tospecify which pixels of a first image are to be replaced by or to becombined with a portion of a second image. The presentation descriptionmay also comprise a mask that defines which areas of the first image areto be combined with or replaced by a second image. The mask may be asingle bit per pixel, as may be used to specify replacement, or maycomprise more than one bit per pixel wherein the plurality of bits foreach pixel may specify the manner in which the images are combined, suchas mix level or intensity, for example. The mask may be implemented aspart of a markup language page, such as HTML or XML, for example. Any ofthe processing methods disclosed herein may further include processesthat produce blurs to match focus or motion blur. Processing methods mayalso include processes to match “graininess” of a first image. Asmentioned above, images are not constrained in format type and are notlimited in methods of combination.

The combination of video signals may employ program code that is loadedinto a set top box and that serves to process or interpret apresentation description and that may provide processing routines usedto combine images and/or audio in a manner described by the presentationdescription. This program code may be termed image combination code andmay include executable code to support any of the aforementioned methodsof combination. Image combination code may be specific to each type ofset top box.

The combination of video signals may also comprise the combination ofassociated audio streams and may include mixing or replacement of audio.For example, an ocean background scene may include sounds such as birdsand surf crashing. As with video images, audio may be selected inresponse to viewer demographics or preferences. The presentationdescription may specify a mix level that varies in time or across aplurality of frames. Mixing of audio may also comprise processing audiosignals to provide multi-channel audio such as surround sound or otherencoded formats.

Embodiments of the present invention may be employed to add content toexisting video programs. The added content may take the form ofadditional description, humorous audio, text, or graphics, statistics,trivia, and the like. As previously disclosed, a video feed may be aninteractive feed such that the viewer may response to displayed imagesor sounds. Methods for rendering and receiving responses to interactiveelements may employ any methods and includes those disclosed inincorporated applications. Methods employed may also include thosedisclosed in U.S. application Ser. No. 10/403,317 filed Mar. 27, 2003 byThomas Lemmons entitled “Post Production Visual Enhancement Rendering”,which is a continuation-in-part of U.S. patent application Ser. No.10/212,289 filed Aug. 8, 2002 by Thomas Lemmons entitled “PostProduction Visual Alterations”, which is based upon U.S. ProvisionalPatent Application No. 60/309,714 filed Aug. 8, 2001 by Thomas Lemmonsentitled “Post Production Visual Alterations”, all of which arespecifically incorporated herein for all that they teach and disclose.As such, an interactive video feed that includes interactive contentcomprising a hotspot, button, or other interactive element, may becombined with another video feed and displayed, and a user response theinteractive area may be received and may be transferred over theInternet, upstream connection, or other network to an associated server.

FIGS. 7 through 25 illustrate, in more detail, the manner in whichenhancements, such as HTML overlay pages, can be used to create embeddedadvertising in video programming.

FIG. 7 is an illustration of one implementation for providing embeddedadvertising using enhancements. Referring to FIG. 7, a scene from avideo broadcast is displayed showing various items or objects in thescene. The video may be displayed on a display device 700, such as atelevision set, which may comprise any type of display such as a plasmadisplay, flat panel display, CRT monitor, or any other display. Thevideo may also be displayed on a computer monitor displaying embeddedads within video streams over the Internet, or the video may bedisplayed on any other type of display system/device including moviesdisplayed at movie theaters or a mobile device. In the scene, a person702 is shown as sitting on a chair 704 at a table 706. The person 702 isseated in front of an object, such as a pizza box 708, from which theperson is eating pizza. The object, or pizza box, may be automaticallyidentified by some existing technique such as pattern recognitionsoftware, as will be discussed in more detail in FIG. 23. Alternatively,tags and markers, as disclosed in the above referenced patentapplications, may be supplied in the video stream that provides theobject identification information (pizza box) and location of the objectin the video stream. The system may then determine if a pizza advertiserhas been listed as an embedded advertiser. That is, a list may becompiled that comprises advertisers who wish to exhibit advertisements,in the form of embedded advertising, in a video presentation. The listof advertisers may also include the item (attribute), which may be inthe form of a product, idea, etc., that the advertisers wish to promote.Such a list of advertisers and attributes may be stored in a database bythe broadcaster. The list of advertisers and attributes is more fullydisclosed with regard to the description of FIG. 9.

Referring again to FIG. 7, once an object, in this instance a pizza box708, is recognized, the advertiser list may be queried for a pizzaadvertiser. If a pizza advertiser has been listed as an embedded adadvertiser, an embedded ad, which may be in the form of a logo, text, orother form of advertisement for the advertiser, is graphically displayed(superimposed or overlaid) on the pizza box. The process of applying, orsuperimposing, the embedded on the pizza box is discussed in furtherdetail with regard to the description of FIG. 23. The embedded ad maycomprise any form of video enhancement, including video sequences, text,graphical images that may be in bitmap format or any other format, orother graphical representation. If the embedded ad is a “hot label,”(e.g., the embedded ad is an interactive enhancement), the viewer maythen click on the hot label to order a pizza from that advertised store.Details of this process are further discussed with respect to FIG. 23.

Embedded ads may comprise enhancements to the original video signal,such as improvements, augmentations or new additions. For example, acompany logo may be applied to a soda-can. The soda-can may alreadyexist within the original video signal, or the soda-can may be insertedinto the original signal. Alternatively, changes may comprise downgradesfrom the original video signal, such as degradations, reductions, ordeletions. For example, a portion of the video may be “erased” in orderto create a blank space, which may be used for applying embedded ads ordisplaying advertisements. Object insertion, in this fashion, can bedone using either video combination techniques, such as disclosed abovein FIGS. 1 through 7, or by using enhancements, such as markup languageoverlay pages, including interactive enhancements.

Embedded ads may be applied to any type of video content and videosignals such as live or taped television broadcasts, animation, filmedvideo and the like or may be captured as a magnetic or optical mediafile, such as in a Digital Video/Versatile/Variable Disc (DVD)feature-length film. Embedded ads may also comprise informationalcontent such as markings. Content may be created by taping or filming,such as the filming of a live broadcast, a feature-length film, atelevision program, or other presentation. Content may also be createdusing animation. Cartoons on television or on feature-length films areone example of animation. Content creation may comprise computeranimation. Computer animation is a principal component of multimediapresentations including feature-length films, and many softwareapplications can be used to create animations that can be displayed onany display device, such as a computer monitor, television screen, ormovie screen.

Of course, embedded ads may be applied to any form of viewing material,including educational programs, multimedia presentations, orentertainment presentations. Such multimedia or entertainmentpresentations may include digitally enhanced or computer enhancedpresentations, animated presentations, which may include feature-lengthanimated films, animated programs that are broadcast over a televisionnetwork, DVD animated presentations, or other forms of animatedpresentations. Additionally, embedded ads may be applied to suchmultimedia or entertainment presentations as feature-length video filmsthat may be shown in movie theatres, perhaps in a digital format, overfiber-optic connections, through laser technology, or through othertechnology.

As mentioned above, techniques to facilitate supplementary content in avideo signal can also be accomplished by combining video signals. Such atechnique is disclosed in U.S. Provisional Application Ser. No.60/278,669, filed Mar. 20, 2001 entitled “DELIVERY OF INTERACTIVE VIDEOCONTENT USING FULL MOTION VIDEO PLANES”, now U.S. Non-Provisionalapplication Ser. No. 10/103,545, filed Mar. 20, 2002 entitled “VIDEOCOMBINER”, both of which are specifically incorporated herein byreference for all that they discloses and teach. In accordance with theabove referenced application, the system is capable of combining videosignals that incorporate embedded ads at the viewer's location. Forexample, multiple video feeds, some of which can include embedded ads,are provided to a viewer's set-top box together with instructions forcombining two or more video feeds. The video feeds can then be combinedin a set-top box or otherwise located at or near the viewer's locationto generate the combined or correlated video signal for display withembedded ads. As mentioned above, the video feeds can be combined by theTV broadcaster prior to transmission over the video distributionnetwork.

Additionally, one or more video feeds can comprise enhanced video thatis provided from an Internet connection. HTML-like scripting can be usedto indicate the layout of the enhanced video signal. Instructions can beprovided for replacement of particular segments comprising the embeddedads space, to make an interactive embedded advertisement.

Hot labels (interactive embedded ads) can appear unobtrusively during avideo broadcast. As disclosed above, the embedded ad itself can be aninteractive element which is placed on specific objects that relate tothe advertising element or in a blank portion of the screen. As alsodisclosed above, embedded ads may not be interactive (e.g., embedded adsmay exist simply as an advertisement or informational material that ispart of the video stream). Insertion of video content to producecombined video streams, as well as insertion of interactive content canbe accomplished by using tags and markers inserted into the video streamas disclosed in U.S. Non-Provisional application Ser. No. 10/076,950,filed Feb. 12, 2002, which is based upon U.S. Provisional ApplicationSer. No. 60/268,350, filed Feb. 12, 2001 entitled “VIDEO TAGS ANDMARKERS”, both of which are specifically incorporated herein byreference for all that they disclose and teach.

Embedded ads that are interactive links can allow for superimposition ofinteractive embedded ad information on products. Embedded ads may appearas a visual enhancement, such as a logo or name, and thus anadvertisement of any party or advertiser, such as a local or nationalcompany, or simply informational. Embedded ads can also beindividualized, localized advertisements. An embodiment may employpattern recognition techniques that provide an automated method ofidentifying objects on which to place embedded ads that pertain to theobject. Embedded ads may be localized to a point where specificadvertisements can be superimposed from set top-boxes, so thatadvertising can be personalized down to the household or individualviewing level.

Embedded ads may be applied to content by creating “embedded ad space,”or space within the content (which may be a video stream), that isavailable for application of an embedded ad. The designation of suchembedded ad space may be completed by manual or automatic methods. Theembedded ad or type of embedded ad to be superimposed may be chosenbased on “attributes,” as further explained with regard to thedescription of FIG. 9. A list of advertisers, along with theadvertisers' associated attributes, may be stored in a database.“Attributes” are a class of actual objects, or blank spaces that may beassociated with an advertiser or with types or classes of information.An embedded ad may be applied, to the embedded ad space within the videostream, according to the class of object or blank space identifiedwithin the video stream. The identified object or blank space (withinthe video stream) may be compared to objects, classes of objects orblank spaces listed as attributes within the aforementioned database.Once a matching actual object or blank space is found, an appropriateembedded ad is chosen and applied to the embedded ad space within thevideo stream.

Referring again to FIG. 7, the embedded ad 710 appears as a logo of alocal franchise called “Paul's Pizzeria.” The embedded ad may be anactual logo, name, or advertisement of a company that can be either alocal company or national chain and may exist as a non-obtrusiveadvertisement that appears to be an actual element of the originallyfilmed scene. The embedded ad 710 appears as a logo on the actual pizzabox, thus appearing completely naturally in the scene. The embedded ad710 may exist as a non-interactive enhancement that is part of the videosignal, and thus the embedded ad may simply exist as an advertisement,or the embedded ad may be an interactive element that has thefunctionality to be activated by the viewer, thus acting as anadvertisement and interactive tool.

Again referring to FIG. 7, if the embedded ad 710 is interactive, theembedded ad may be activated by the viewer 712. As previously described,the “viewer” is a person watching the video. Activation of the embeddedad 710 may occur in several ways, including toggling through a series ofembedded ads using control buttons on some form of a hand-held controldevice 716, such as a remote-control, activating a cursor 718 using awired or wireless mouse, or by other means including but not by way oflimitation, light beams, such as IR beams that can be used as pointersto point at embedded ads. Such a system may operate in a fashion similarto a touch screen that identifies the light beam rather than a user'sfinger. A cursor 718 may be an indicator, such as a spot, a flashingline, an arrow, a symbol, or any other indicator, that is displayed onthe TV screen 700. The cursor position may be controlled by another formof a hand-held control device, such as a wired mouse or a wireless mouse(not shown) in a similar fashion to that of a mouse being connected to acomputer and monitor. Once activated, the embedded ad may make availablea host of options, including bringing up an order form 720, which mayallow the viewer to begin the process of directly ordering a pizza. Theorder form 720 may appear on an unused portion of the TV screen 700,thereby causing no interruption of the current video presentation.

FIG. 8 is an illustration of the manner in which interactive content maybe displayed. Once activated, the embedded ad (as referred to in FIG. 7)may display information in an enhanced content window or border area801. The current video presentation shown on display device 800 mayremain viewable in the broadcast window 840. As shown in FIG. 8, thedisplayed video that includes the same objects that appeared in FIG. 7,including the table 826, pizza box 828, embedded ad 830, person 832, andchair 834, remain visible. Alternatively, the information brought up bythe embedded ad may be displayed on a completely separate page. Theembedded ad may bring up a graphical user interface (GUI) 802, a textbox, or some other presentation scheme that serves to presentinformation in some manner to the viewer. The GUI 802 may presentadditional information 804 about the company for which the embedded adhas been activated. Such additional information may be submitted by theadvertiser or by the author (e.g., person examining and/or editing thevideo signal). The additional information 804 may be contained in aseparate storage device, such as a database. The database containingadditional advertiser information is more fully disclosed with regard tothe description of FIG. 9. Once an embedded ad is activated, thiscorresponding database can be accessed for the additional information.The separate database may be located in a central computer, or onmultiple computers, which may be a server at the head-end, aretransmission site, other site, or in the household of the viewer in aset-top box. The central computer is more fully disclosed with regard tothe description of FIGS. 17 and 18. The central computer may insert theembedded ad. Insertion of the embedded ad is discussed in further detailin FIG. 23.

Referring again to FIG. 8, the additional information 804 may includecompany profile 806, web address 808, coupons 810, or other information.The web address 808 may simply be an address that is non-interactive andis simply there for the viewer to take note of and to have theopportunity to visit that address at a later time, or the web addressmay act as a direct, interactive link (e.g., hyperlink). The interactivelink may link the viewer to the company home page, or the companywebsite that allows the viewer to make a purchase. Furthermore, a directpurchase link (e.g., order pizza link 814) takes the user to an “order apizza” website 814, which may be a special website designed specificallyfor viewer interaction with embedded ads. In addition, if the advertiseris a national company, the additional information may include localdistributors or franchises 812 of the national company. The embedded admay provide a link to the local server to provide information regardingthe local store. For example, if Paul's Pizzeria is a national chain,the local broadcaster may superimpose the local franchise of Paul'sPizzeria at the server at the head-end, as illustrated in FIG. 17. Theviewer, when clicking on the embedded ad, will see their local Paul's'Pizzeria in the additional information display and be able to order apizza directly from their local store.

Furthermore, the viewer may also have the choice of accessing links thathave nothing to do with the current video presentation. For example, ona different part of the graphical user interface 802, the viewer may beable to activate links 816 that facilitate access to the world-wide-web(WWW). For example, the viewer may be able to cast a vote 818 on whichcharacter the viewer liked the best during a particular televisionprogram such as the “Survivor” series. The viewer may also be able tocast a vote 820 for elections, read about their local politicalrepresentatives by clicking on an interactive link 822 and enteringtheir local information, or even read about current world events 824.Other information may also be accessed via these WWW activate links 816.

FIG. 9 is a graphical representation of a database that provides a listof additional information associated with advertisers, as listed indatabase 900. As previously mentioned with regard to the description ofFIG. 8, the additional information of an advertiser may be contained ina storage device, such as a database 900. As illustrated in FIG. 9, suchadditional information may be submitted as an entry 902. For example,one entry may be for the advertiser's name, Paul's Pizzeria 901, withadditional associated information that may include company profile, webaddress, coupons, or other information. The additional information maybe submitted by the advertiser and/or by the author. Additionally, otheradvertisers may be stored in the same database along with similarinformation, as a separate entry 904.

FIG. 10 is a graphical representation of a database that provides a listof attributes associated with advertisers, as listed in database 1006.Database 1006 may also contain an embedded ad itself, or a link to anembedded ad. Referring to FIG. 10, in order to paint the embedded adonto a particular object or even onto a blank space, there is adetermination of whether or not there is an appropriate embedded ad toapply. Thus, a list of advertisers is stored in a storage device, suchas a database 1006, along with associated attributes. “Attributes” aredefined as any actual object or categories of objects that areassociated with the advertiser, or “blank space,” which willsubsequently be explained. Attributes may be submitted by the advertiserand/or by the author. Referring to FIG. 10, an advertiser, as may beseen as one entry out of many, is stored in the database 1006. Forexample, the advertiser General Electric 1014 may be an entry 1008.General Electric may be accompanied by a list of attributes 1010. Suchattributes 1010 may include actual objects, such as “stove,”“refrigerator,” “toaster,” and other actual objects or categories suchas appliances, etc. Alternatively, advertisers may wish to have theopportunity to advertise on an available blank space; in this case, theadvertiser may submit “blank space” as one of their attributes. Forexample, the advertiser Joe's Appliances 1016, as seen in entry 1012,may elect to list “blank space” 1014 as an attribute, which would allowJoe's Appliances the opportunity to bid on a blank space in the videocontent as well as to apply an advertisement to an actual object. Blankspace may be further constrained to specific areas such as the dasherboards below the Colorado Avalanche hockey bench. Blank spaceadvertising is more fully disclosed with regard to the description ofFIGS. 11 and 12. Additionally, other advertisers may be stored in thesame database along with similar information, as other entries such as1018 and 1020. In addition, this database may store the file name ofand/or link to an embedded ad graphics file, which may be stored in aseparate location. Alternatively, the database may contain embedded adtext files 1026, 1028. These concepts are more fully disclosed withrespect to the description of FIGS. 13 and 14. The manner in which acompany is chosen as an advertiser using an embedded ad is discussed ingreater detail in FIG. 24. If the system of the present invention findsa pizza advertiser in the list of advertisers, an embedded ad is paintedonto the identified pizza box, at which time the viewer may click on theembedded ad and order a pizza. The process of applying the embedded adis discussed in greater detail in FIG. 23.

FIGS. 11 and 12 are illustrations of scenes showing other embodiments.Embedded ads can be coded to change on a per-scene basis. As illustratedby FIG. 11, in scene one 1100 of a television show, a refrigerator 1102may be superimposed (e.g., overlaid) with an embedded ad 1104. In scenetwo 1206, the camera angle is different, which changes the position ofthe refrigerator 1208. During this scene, the embedded ad 1210 remainson the refrigerator. Furthermore, instead of the embedded ad appearingas a normal part of the scene as illustrated by embedded ads 1104 and1210, an embedded ad 1212 may appear larger and more apparent, such astext written outside the boundaries of the area rug 1214, thus appearingto the viewer as an obvious part of the scene.

FIGS. 13, 14 and 15 are illustrations of scenes showing otherembodiments. As shown in FIG. 13, a football game is displayed on adisplay screen 1300. The scene illustrates a football field 1302 andviewing stands with real banner ads 1304, 1306 and 1308 as typicallyfound in sports arenas. Embedded advertising is supplemented in thispre-existing video signal by superimposing embedded ads 1310, 1312, and1314 on to the playing field 1302. Thus, the final enhanced video signalappears to the viewer as though the field embedded ads 1310, 1312, and1314 are inherent to the setting and look as if they are a natural partof the scene.

In FIG. 14, a display screen 1430 is shown wherein embedded ads 1434,1436, and 1438 are superimposed over the top of the real banner ads thatexisted in the un-enhanced video signal. The enhanced video signalcompletely obscures the real banner ads present at the sports arena andappears to the viewer as though the superimposed embedded ads 1434,1436, and 1438 are inherent to the setting and are a natural part of thescene. In this way, local embedded advertising can supercede nationalembedded advertising for different regions of a nationally broadcastprogram. FIG. 14 also illustrates an informational embedded ad 1440placed on the football field 1432 denoting the end zone of the team onoffense.

In FIG. 15, a display screen 1548 is shown wherein interactive playername embedded ads 1550, 1552, 1554 and 1556 are superimposed over thehelmets of the football players in a game. In this instance, theenhanced video signal is not viewed as inherent to the setting and theembedded ads may be used as hot labels to bring up additional playerinformation when accessed. This information can include personal,biographical or statistical data or the like for the accessed player butrequires the user to click on the embedded ad.

FIGS. 16 and 17 are illustrations of scenes showing other embodiments.Each time that an embedded ad appears in a new scene, the embedded admay be a different advertisement. As shown in FIG. 16, scene one 1602,the person 1604 reaches into an open refrigerator 1606 for a soda can1608. The embedded ad 1610 is a “Coke” logo. In FIG. 17, scene two 1712,the same person 1714, while standing beside the closed refrigerator1716, drinks from the soda can 1718. The embedded ad 1720 is now a“Pepsi” logo. Applying embedded ads in this fashion may be done throughpattern recognition and localized superimposition of ads, as willsubsequently be explained.

FIG. 16 also illustrates the addition of an interactive tattoo embeddedad. In scene one 1602 the person 1604 is depicted with tattoo containingnationally recognized logo 1650 superimposed on the persons arm. In thisscene, the tattoo is overlaid with a transparent background to look likea natural part of the scene. Accessing this interactive hot label maylink the viewer to either the home page of the company depicted in thenationally recognized logo or to an address and phone number of a localtattoo shop.

FIG. 18 is a schematic block diagram that generally illustrates anotherembodiment. As shown in FIG. 18, an origination site 1800 may providecontent. Content comprises an un-enhanced video signal 1802. The contentmay be created by a content provider such as a television network,television studio, a live broadcast, a server providing code or datasuch as an Applications Service Provider, an Internet Service Provider,or any other type of content provider that may be capable of sendingcontent to viewers. The origination site 1800 may be capable ofgenerating the content itself, or may simply provide content that issupplied from an outside source 1804 in the form of an un-enhanced videosignal 1806. The origination site 1800 transmits the un-enhanced videosignal 1802 to a server 1808. The address of the un-enhanced videosignal can be provided by the advertiser to the TV broadcaster.

Referring to FIG. 18, the server 1808 interacts with a central computer1810 and applies an embedded ad 1812 to the un-enhanced video signal1802. This embedded ad 1812 may comprise any form of video enhancement,including video sequences, text or graphical images and may be in anyformat or other graphical representation such as bitmap or GIF. Theserver 1808 accesses one or more of the multiple data files 1822contained in a central computer 1810. There may be one data file foreach advertiser or one data file for each advertisement (e.g., embeddedad) and may comprise an embedded ad 1812 that exists as graphicalinformation within the data file. The data file 1822 may also compriseplacement and contour data 1814 which may be obtained by use of existingcomputer recognition, physical manipulation, or other techniques. Byusing placement and contour data, the embedded ad either may be appliedunobtrusively to the video signal, or may appear to be an obviousaddition to the original video signal 1802. Alternatively, the data file1822 may comprise secondary video streams that may be superimposed(overlaid) onto the original un-enhanced video signal 1802.

Referring again to FIG. 18, the central computer 1810 may also contain adatabase 1816, as referred to in the description of FIG. 9, which storeseach advertiser name and additional information. The central computer1810 may also contain another database 1818, as referred to in thedescription of FIG. 10, which stores each advertiser name, associatedattributes as well as a link 1022, 1024 to the embedded ad (asillustrated in FIG. 10) or an embedded ad text file 1026 and 1028. Eachentry for each advertiser in the database 1818 (as previouslyillustrated in FIG. 10) may be linked to the associated advertiser'sdata file 1822 though a link 1820. Again, the databases and the datafiles may be stored in a central computer or on multiple computers whichcan comprise a server located at a head-end or in a set-top box.

Referring to FIG. 18, the server 1808 may incorporate existing patternrecognition software and techniques to provide one way in whichavailable advertising space may be identified. However, identificationof advertising space can be achieved by several methods as may befurther seen with respect to the description of FIG. 25. The patternrecognition software may communicate with the attribute database 1818,stored in the central computer 1810. As previously discussed, theattributes may be actual objects or “blank space” attributes. Thepattern recognition software may search the unenhanced video stream 1802for these actual objects or blank spaces. Once a pattern is identified,the server searches the attribute database 1818 for a matchingattribute. In the event that more than one company has listed theidentified actual object or blank space as an attribute, bidding maystart to choose which advertiser will receive the embedded ad. Adetailed discussion of criteria, process and application of the embeddedad is described in further detail with respect to FIG. 23.

Referring again to FIG. 18, once the server 1808 has applied theembedded ad 1812 which the server accessed from either the data file1822, or the attribute and text-embedded ad database 1818, the outputfrom the server is an enhanced video signal 1824; for example, a videosignal which has one or more embedded ads applied to the video signal.The enhanced video signal 1824 is transmitted via a link 1826 to aset-top box 1828. The link 1826 may comprise a radio frequency link,satellite link, cable connection, or any other type of communication.The set-top box 1828 receives and transmits the enhanced video signal1824 to a display device 1830, such as a computer monitor or TV screen.

FIG. 19 is a schematic block diagram that generally illustrates anotherembodiment. In FIG. 19, the embedded ad is applied by the set-top box,rather than the server. Content comprises an un-enhanced video signal1902. The content may be created by a content provider such as atelevision network, television studio, a live broadcast, a serverproviding code or data such as an Applications Service Provider, anInternet Service Provider, or any other type of content provider thatmay be capable of sending content to viewers. The origination site 1900may be capable of generating the content itself, or may simply providecontent that is supplied from an outside source 1904 in the form of anun-enhanced video signal 1906. The origination site 1900 transmits theun-enhanced video signal 1902 via a link 1926, which again, may comprisea radio frequency link, a satellite link, or other type of link, to theset-top box 1928. The set-top box 1928 accesses the central computer1910 to apply one or more embedded ads to the un-enhanced video signal1902 to produce an enhanced video signal 1924 output. This enhancedvideo signal 1924 is then sent from the set-top box 1928 to a displaydevice 1930, such as a computer monitor or television screen.

FIG. 20 is a flow diagram that illustrates the steps performed inapplying embedded ads to a video stream 2000. As shown in FIG. 20,content is created at step 2002. As previously disclosed, the contentmay be created by a content provider such as a television network,television studio, a live broadcast, a server providing data such as anApplications Service Provider, an Internet Service Provider, or anyother type of content provider that may be capable of sending content toviewers. The process is continued at step 2004 where availableadvertising space is designated by manual methods. Such manual methodsmay comprise having an author, such as a person examining theun-enhanced video signal, manually designate places within the currentunenhanced video signal that are available for advertising.Alternatively, the process may proceed from step 2002, where the contentis created, to step 2006, where the designation of available advertisingspace is automated. Automatic methods of designating advertising spacemay include visual (pattern) recognition or other techniques. Theprocess proceeds to step 2008, where a list of the available advertisingspaces is generated. This list may be generated by a server that alsocontains a list of advertisers and their associated attributes. At step2010, the available advertising space that has been recently designatedis bid upon by the advertisers. At step 2012, embedded ads are placed inthe video while the video content is being shown on a display device,such as a computer monitor or television screen.

Embedded ads may be representative of local or national companies andmay be superimposed on a per-household basis. This individualizedsuperimposition may be facilitated by utilizing affinity data, such asdisclosed in U.S. patent application Ser. No. 10/046,618, filed Oct. 26,2001 entitled “Collection of Affinity Data from Television, Video, orSimilar Transmissions,” which is based upon U.S. Provisional ApplicationSer. No. 60/243,506, filed Oct. 26, 2000 entitled “Collection ofAffinity Data from Television, Video, or Similar Transmission,” and alsodisclosed in U.S. Non-Provisional application Ser. No. 10/099,054, filedMar. 13, 2002 entitled “Affinity Marketing for Interactive MediaSystems,” which is based on U.S. Provisional Application Ser. No.60/275,295, filed Mar. 13, 2001 entitled “Affinity Marking forInteractive Television,” all of which are specifically incorporatedherein by reference for all that they discloses and teach. Embedded adsmay also be superimposed on a house-by-house basis by collecting datareceived from personalized remote controls, as disclosed in U.S. PatentApplication Ser. No. 60/227,916 and Ser. No. 09/941,148 referencedabove. In accordance with the above applications, affinity data providesinformation to a television broadcaster as to whether or not an audienceis actually engaged in the current video presentation, through use of avariety of sensors, and feedback mechanisms as opposed to simply havingthe television on and not watching the television. The personalizedremote is used to collect data regarding what shows are being watched byeach different member of a household. Together, the use of technologyfrom affinity data and personalized remotes may allow for personaltargeting of advertisements. In that regard, television broadcasters mayprovide data to advertisers as to how many households will receive theirad prior to starting the bidding process. This information may be storedwith the attribute data and transmitted to the advertiser when thetelevision broadcaster solicits bids.

FIG. 21 is a flow diagram illustrating the steps for carrying out theinstallation of embedded ads by manual designation of availableadvertising space 2100. The content or video signal is displayed in step2101. In step 2102, the author, previously defined as the personexamining the video signal, examines the video signal. The processproceeds to step 2104 where the author manually designates availableadvertising space within the un-enhanced video signal. That is, theauthor looks for possible places existing in the original unenhancedvideo stream onto which to place advertisements. Such possible placesmay include actual objects, onto which a particular logo of a companymay be applied or blank spaces onto which any advertiser desiring toplace their advertisement on a blank space may utilize. In addition, theauthor may plan the insertion of objects such as billboards, cans, cars,clothing, etc. In that regard, the existing objects in the program maybe changed. For example, the video signal may show an actress driving aChevy. The embedded advertising has the ability to change the Chevy to aFord. Once the author identifies an embedded advertising opportunity,the author designates the identified opportunity and placement spot asbeing available for an embedded advertisement. Proceeding to step 2106,placement and contour data for the identified object, an insertedobject, or blank space, which can act as the available advertisingspace, is generated. The placement/contour data may be obtained by useof existing computer recognition techniques, physical manipulation, orother techniques, and may be generated by a computer located at thehead-end, the set-top box, or other site. Utility of the contour datamay include making the embedded ad appear more naturally in the scene.As an example, if the object to which the embedded ad is applied isround, the embedded ad, as text or graphics or any visual enhancement,may be wrapped around the object's physical shape so that the embeddedad-enhanced object appears to look natural. By generatingplacement/contour data, the available advertising space is designatedand comprises a boundary around the identified object.

Again referring to FIG. 21, in step 2108, the placement/contour data iscombined with the video signal. In step 2110, the combinedplacement/contour data and video signal are accessed by the computer.The location of the computer may be at the head-end (transmission site),retransmission site, the set top box (household), or some other site. Instep 2112, the embedded ad is applied to the unenhanced video signal. Aspreviously disclosed, the embedded ad may be applied by overlaying data,which may contain the embedded ad itself along with placement/contourdata, onto the original un-enhanced video signal. The data may be in theform of a Hypertext Mark-up Language (HTML) overlay page, which ismostly blank except for the embedded ad that the HTML overlay pagecontains. In this way, only the embedded ad is visible as the videosignal flows through the system. The server or set-top box overlays theHTML page onto the video signal, thus applying the embedded ad toselective areas of the video signal by using the coordinates given bythe placement/contour data. The embedded ad may be color-coded orpartially transparent such that the embedded ad appears to be part ofthe original broadcast to the viewer. Alternatively, the embedded ad maybe presented as an obvious addition to the original video signal, as waspreviously illustrated in FIG. 11.

The embedded ad may constitute graphical, image or text information andmay be in the form of an MPEG stream, a bitmap, a JPEG image, etc. Thegraphical, image, or text information of the embedded ad may be storedlocally in the set-top box, at the head-end, at a distribution point ormay be accessed over the Internet or other network. As indicated above,an HTML overlay page may be used to combine the information of theembedded ad with the pre-existing video stream. However, any type ofcomputer-generated bitmap can be used to produce a screen that can becombined with the pre-existing video signal in one or more frames.

In addition, the computer-generated bitmap need not just overlay thepreexisting video image. For example, the combination of the video imageand the embedded ad image may employ arithmetic or logical combinationsof the bitmap information. Such combinations may employ rasteroperations common to windowed graphical user interface environments suchas Microsoft Windows™. These arithmetic and logical combinations allowthe embedded ad to replace or partially replace the pre-existing videoimage with the content of the embedded ad. Half toning and “see-through”embedded ads can also be produced using such techniques. The resultantvideo signal that is displayed to the viewer allows the embedded ad tothen appear as an integral part of the displayed image. Hence, theprocess of superimposing the embedded ad content on the pre-existingvideo content can be accomplished by simply overlaying a computergenerated bitmap on the pre-existing video content, which may alsocomprise bitmap information generated from an MPEG type of signal.Alternatively, the process of superimposing the embedded ad content onthe pre-existing video content can be accomplished by actually combiningthe computer generated bitmap containing the embedded ad content withthe pre-existing video content such as with arithmetic or logicalcombinations, as disclosed above. Further, commonly used color keyingtechniques, known in the television and computer arts and commonlyreferred to as “blue screen” and “chroma-key”, can also be employed. Forexample, the embedded ad content may include specific colors that havebeen designated to not replace the content of the pre-existing videocontent, or specific colors may be employed to define areas ofreplacement or combination. In this fashion, the “transparency” of theembedded ad content can be accomplished using color key techniques.

The process of selecting the embedded ad content can be accomplished invarious ways. For example, the embedded ad content may be accessed overa network connection. This network connection may be based uponpreferences stored in a set-top box or based upon a tag or marker thatis transmitted as part of the video content of the pre-existing videostream. In addition, the network connection may be an upstreamconnection to a server, such as a server located at the head-end or somedistribution point that has stored embedded ad content that can then bedownloaded to the set-top box either through the vertical blankinginterval (VBI) or an out-of-band channel. In addition, the embedded adcontent that is accessed in this fashion may be based upon preferencesthat are stored either at the head-end, an access point or at theset-top box. In addition, the bitmap information of the embedded adcontent may be generated from instructions that are downloaded from anetwork such as the various network connections described above. Theseinstructions may comprise executable code that is capable of generatinggraphics or text at the set-top box level. Again, these can be basedupon preferences.

While the foregoing description may employ references to a set-top box,it is specifically intended that any and all such description shallapply to any receiving unit operable to provide a similar function,including intelligent televisions, multimedia computers, satellitereceivers, etc., and also applies to portable systems that may employwireless networks to receive video transmission and that may employwireless networks to access embedded ad information.

FIG. 22 is a flow diagram illustrating the steps for carrying out theinstallation of embedded ads by automatic designation of availableadvertising space and searching for actual objects 2200. In previousembodiments of the invention, the designation of available advertisingspace was performed by manual methods. Alternatively, the designation ofopportunities for embedded advertising may be accomplishedautomatically. The following steps outline an automated process ofdesignating available advertising opportunities. In step 2201, thecurrent unenhanced video signal is displayed. The process proceeds tostep 2202 where pattern recognition techniques may be applied to thecurrent unenhanced video presentation. These pattern recognitiontechniques may be performed by a computer at a head-end (transmissionsite), retransmission site, the set top box (e.g., household), or someother site. In step 2204, a query is made as to whether a pattern hasbeen recognized. Recognized patterns may include actual objects. If apattern has not been recognized, the process returns to step 2202 andcontinues applying the pattern recognition component to the currentvideo presentation. Returning to step 2204, if some sort of pattern isrecognized, the process proceeds to step 2206, where a query is made asto whether the recognized pattern is an actual object. If the recognizedpattern is not an actual object, the process returns to step 2202 andcontinues employing the pattern recognition component. After the queryat step 2206, if it is determined that the recognized pattern is anactual object, the process proceeds to step 2208 where placement andcontour data is generated, which may include coordinates of therecognized object. As previously disclosed, placement and contour datamay be generated by use of existing computer recognition techniques,physical manipulation, or other techniques. The placement/contour datamay be generated by a computer located at the head-end, the set-top box,or other site. By generating placement/contour data, the availableadvertising space is designated and comprises a boundary around theidentified object. Recognized objects can also be replaced automaticallyby other objects, as described above.

Referring again to FIG. 22, the placement/contour data is combined withthe video signal in step 2210. Thus, the output signal after step 2210is an enhanced video signal. The placement/contour data and videosignal, collectively referred to as the enhanced video signal, areaccessed by the computer in step 2212. As previously disclosed, thelocation of the computer may be at a head-end (transmission site),retransmission site, the set top box (household), or some other site. Instep 2214, the computer overlays a HTML page which is blank except forthe embedded ad, onto the video signal. Thus, one or more embedded adsare applied to the selective areas of the video signal by using thecoordinates given by the placement/contour data.

FIG. 23 is a flow diagram illustrating the steps for carrying out theinstallation of embedded ads by automatic designation of availableadvertising space and searching for blank spaces 2300. The principlesdiscussed in FIG. 23 are similar to the principles discussed in FIG. 22,with the pattern recognition component searching for a blank spaceinstead of an actual object. As previously disclosed in FIG. 22, thedesignation of available advertising space is completed by use ofautomated methods. The process begins at step 2301 where the unenhancedvideo signal is displayed. The process continues at step 2302, wherepattern recognition techniques may be applied to the unenhanced videosignal. Again, the pattern recognition techniques may be performed by acomputer at a head-end (transmission site), retransmission site, the settop box (household), or some other location. At step 2304, a query ismade as to whether a pattern has been recognized. Recognized patternsmay include blank spaces. If a pattern has not been recognized, theprocess returns to step 2304 and continues applying the patternrecognition component to the current video presentation. Returning tostep 2304, if a pattern is recognized, the process proceeds to step2306, where a query is made as to whether the recognized pattern is ablank space. If the recognized pattern is not a blank space, the processreturns to step 2302 and continues employing the pattern recognitioncomponent. After the query at step 2306, if it is determined that therecognized pattern is a blank space, the process proceeds to step 1208where placement and contour data is generated and may includecoordinates of the recognized object. As previously disclosed, placementand contour data may be generated by use of existing computerrecognition techniques, physical manipulation, or other techniques by acomputer at the head-end, set-top box, or other site. By generatingplacement/contour data, the available advertising space is designatedand comprises a boundary around the identified object.

Referring again to FIG. 23, the placement/contour data is combined withthe video signal in step 2310. The resultant enhanced video signal isaccessed by the computer in step 2312. As previously disclosed, thelocation of the computer may be at the head-end (transmission site),retransmission site, the set top box (household), or some otherlocation. At step 2314 the computer overlays a HTML page, which is blankexcept for the embedded ad, onto the video signal. Thus, one or moreembedded ads are applied to the selective areas of the video signal byusing the coordinates given by the placement/contour data.

FIG. 24 is a flow diagram illustrating the steps for carrying out theinstallation of embedded ads, using automatic methods, in a manner thatcombines the processes of looking for an actual object while searchingfor a blank space simultaneously 2400. As shown in FIG. 24, the authorcreates a list of advertisers at step 2401. At step 2402, attributes areentered and assigned to respective advertisers. As previously defined,attributes are any actual object that is associated with the advertiseror a blank space. Actual object attributes and blank space attributesare illustrated in FIGS. 11-15. As previously discussed in FIG. 8, eachadvertiser may be accompanied by a list of additional information.

Referring again to FIG. 24, at step 2404 a database record of eachcompany and their associated attributes, as illustrated in FIG. 10, iscreated. This record is saved to the database at step 2406. At step2408, the video signal, such as a television sitcom, is played. At step2410 a pattern recognition component is activated, which constantlysearches the video content to identify patterns, e.g., actual objects orblank spaces. At step 2412, a decision is made as to whether or not apattern has been recognized. If a pattern has been recognized, theprocess proceeds to step 2414. If a pattern has not been recognized, theprocess returns to step 2410 and continues searching the video signalfor additional patterns. If a pattern has been recognized at step 2414,a decision is made as to what type of pattern is recognized. If theidentified pattern is a blank space, the process proceeds to step 2416.At step 2416, placement and/or contour data is generated which mayinclude coordinates of the identified object. As previously disclosed,the placement and contour data may be generated by use of existingcomputer recognition techniques, physical manipulation, or othertechniques. By generating placement data, available advertising space isdesignated. At step 2418, the database is searched for an appropriateadvertisement. If no appropriate advertisement is found, the processreturns to step 2410 and continues searching the video signal. At step2420, the advertisers with matching attributes bid on the ad and thewinning bidder is chosen.

Referring back to FIGS. 13-15, the pattern recognition component mayrecognize a blank space, such as arena panels, open ice or field, etc.,onto which multiple advertisers may want to advertise (e.g., multipleadvertisers had listed “blank space” as one of their attributes). Anadvertiser may have chosen an attribute such as a hockey game, baseballgame, Friends episode, Seinfeld episode, situation comedy, prime timebroadcast, rerun, etc. to be placed in the bidding. This allows theadvertisers to be placed in the bidding process for areas of specificinterest and allows the broadcasters to target advertisers in placingbids on topics, times, programs, etc. of interest to the advertisers.

Referring again to FIG. 24, at step 2422, Extensible Mark-up Language(XML) code is created and superimposed (e.g., overlaid) onto thedesignated blank space within the unenhanced video signal. The XML codeis superimposed onto the unenhanced video signal by use of theplacement/contour data previously acquired in step 2416. In the finalstep 2424, code is translated, via an Extensible Scripting Language(XSL) parser, from XML into code for Web TV set-top boxes, HTML, JavaScript (JS) (code for Liberate set-top boxes) and/or code for otherset-top boxes.

Referring again to FIG. 24, the process returns to step 2414. If thepattern recognized is an actual object, the process proceeds to step2426. At step 2426, placement and/or contour data is generated. Again,the placement and contour data may be generated by use of existingcomputer recognition techniques, physical manipulation, or othertechniques. At step 2428, the database is searched for a matchingattribute. If no matching attribute is found, the process returns tostep 2410 and the pattern recognition program continues searching thevideo stream. If a matching attribute is found, the process proceeds tostep 2430. At step 2430, the advertisers that have matching attributesare contacted to place a bid. The information regarding the embedded adis provided to the advertisers. The advertisers bid on the embedded adand the winning bid is chosen. If there are no advertisers with matchingattributes, the broadcaster may wish to categorize the opportunity as ablank space, which means that the object will be removed and a new oneinserted. This provides move opportunities to obtain bidders for theembedded advertising opportunities.

The process then proceeds to step 2432, where XML code is created andsuperimposed on the designated actual object or replaces the object. Inthe final step 2434, code is translated, via an XSL parser, from XMLinto code for Web TV set-top boxes, HTML, JS (code for Liberate set-topboxes) and/or code for other set-top boxes.

FIG. 25 is a flow diagram illustrating the steps performed for theinstallation of embedded ads by using tags and markers to describeadvertising space 2500. As mentioned previously, pattern recognitiontechniques may provide a way in which available advertising space may beidentified. However, identification of advertising space can be achievedby several methods. For example, available advertising space within thevideo stream may be defined by use of technology as disclosed in U.S.Application Ser. No. 60/268,350 and Ser. No. 10/076,950 referencedabove.

Referring again to FIG. 25, content is created at step 2501. Proceedingto step 2502, available advertising space is designated. As discussed inFIGS. 21 and 22, the designation of available advertising space may beachieved by automatic or manual methods. If the available advertisingspace has been designated by automatic methods, it can then be checkedand edited by manual methods. The process proceeds to step 2504 wheretags are generated as disclosed in the above references. These tagsdescribe the advertising space that is designated in step 2502. Aspreviously disclosed, the advertising space may be an actual object or ablank space, and is referred to as an attribute. The tags define thelocation and perhaps contour of the identified object. That is, the tagsmay comprise a location or several defining locations, perhaps in x andy coordinates, and a contour of the object or blank space (attribute)identified. The tag may flag the attribute as being a blank space or anactual object. This placement and contour data may also be generated byuse of existing computer recognition techniques, physical manipulation,or other techniques. The process proceeds to step 2506, where the tag isplaced in the video stream.

Referring again to FIG. 25, the process proceeds to step 2508 wheremarkers are generated in a manner disclosed above. These markersindicate the position of a division between each video segment of thevideo stream. That is, the markers indicate the beginning and the end ofeach video segment within the video stream. The process proceeds to step2510, where the markers are inserted in the video stream. This can bedone by the content provider or marketer of the program.

At step 2512, a “moving location” tag may be superimposed into the videostream and that indicates the new location of an object on the screen ina manner such as disclosed in U.S. Provisional Application Ser. No.60/354,745, filed Feb. 6, 2002 entitled “TRACKING MOVING OBJECTS ONVIDEO WITH INTERACTIVE ACCESS POINTS”, which is specificallyincorporated herein by reference for all that it discloses and teaches.

For example, an identified object, which may be refrigerator, may moveslightly from one frame to the next. Such a tag would account for therefrigerator's new location and/or orientation, and this tag maycomprise new x and y coordinates and contour data of the refrigerator.The tags and markers may be sent during the video's vertical blankinginterval (VBI). Any known standard for transmitting enhanced data,including advanced television enhancement forum (ATVEF), may be used.References made herein to the ATVEF specification are made forillustrative purposes only, and such references should not be construedas an endorsement, in any manner, of the ATVEF specification.

Referring again to FIG. 25, the process proceeds to step 2514, where thevideo signal, which has been enhanced with tags and markers, istransmitted to a broadcaster or to the head-end where the tag is read.Alternatively, the broadcaster could read the tag, if the advertisementto be superimposed is a national advertisement. Referring again to step2514, the head-end reads the tag marking the location of an object orblank space and providing other information such as object or blankspace profile, size, type, or other information. The process thenproceeds to step 2516 where the head-end queries a list of advertisers,as referred to in FIG. 3. A query is made as to whether the listcontains an advertiser that has listed the identified object or blankspace as one of the advertiser's attributes. At step 2518, advertisersthat have matching attributes are selected for bidding and the owningbidder is selected. The embedded ad is consequently superimposed intothe video stream at head-end at step 2520. The display size of theembedded ad may be determined by the coordinate and contour dataprovided by the tag.

Various embodiments of the present invention therefore provide embeddedadvertising that can be bid upon by advertisers and implemented locally,or even on a household basis, which is the ultimate form of localizedadvertising. Localization of advertisements can be extremely effectiveespecially when combined with personal preference and demographic data.Furthermore, by adding embedded ads to video content in apost-production environment, local advertising can be provided in asimple and inexpensive manner. Targeted advertising in this manner canbe very valuable to prospective advertisers, and advertisers will bewilling to bid high prices for such advertising. As such, revenue ofbroadcasters may be increased. Embedded ads may be unobtrusive,non-intrusive, and do not compromise size, color, or continuity of thevideo content. Embedded ads that are interactive may be activated byseveral methods, including clicking on the embedded ad with a hand-helddevice or remote, moving a pointing device over the embedded ad, orother methods. Embedded ads can transfer information back to thehand-held device for display/interaction, perhaps by a wireless orinfrared link, send information back to a server for data aggregation,or perform any number of other actions. Further, a personalizedhand-held device can be used in accordance with the present inventionthat allows the system to provide personalized information and allowsseveral people to interact with the same program simultaneously. Suchprinciples are disclosed in U.S. Non-Provisional application Ser. No.09/941,148, filed Aug. 27, 2001 entitled “Personalized Remote Control,”which is based upon U.S. Provisional Application Ser. No. 60/227,916,filed Aug. 25, 2000 entitled “Personalized Remote Control,” both ofwhich are specifically incorporated herein by reference for all thatthey disclose and teach.

Embedded ads encourage impulse buying of products or services whichencourages advertisers to bid on embedded advertising. Embedded ads forproducts and services may be directly related to the content beingwatched or may provide additional information for the purchase of itemsthat have no relevance to the video content. Embedded ads can betailored individual one-on-one advertising, allow the viewer to vote,perform other interactive responses, or facilitate access to theworld-wide-web. Embedded ads can display information in a number ofdifferent ways, such as on a relevant item, on a separate page, or on anunused portion of the video presentation. Traditional enhancements, suchas a window or text-box on the side or bottom of the television screenmay appear once the embedded ad is activated. A special small icon,perhaps called a “Buy-Con™,” may appear in the corner of screenindicating to the viewer the availability of embedded ads in the scene.

Embedded ads may be installed automatically by use of existing patternrecognition technology. Patterns that are identified by the software maybe actual objects that exist in the current video stream, or blankspaces that exist in the current video stream. Actual objects or blankspaces may be an appropriate place to superimpose an embedded ad. Thepatterns identified by the pattern recognition software are matched withthe list of attributes to identify an applicable embedded ad, and thepatterns are assigned placement and/or contour data. Theplacement/contour data is then used by a computer to apply the embeddedad in the current video stream. Objects can be replaced with otherobjects having the same or different attributes.

Alternatively, the installation of embedded ads may be achieved bymanually installing embedded ads into the programming content such asdisclosed in U.S. patent application Ser. No. 10/041,881, entitled“Creating On Content Enhancements,” which is based upon U.S. ProvisionalPatent Application Ser. No. 60/243,107, entitled “Creating On ContentEnhancements,” referred to and incorporated above. Initially, manuallyinstalling advertising into the programming content may be done at thehead-end (transmission site), at a retransmission site, or in individualhouseholds according to their demographics. The principles of placingembedded ads on a household-to-household basis are disclosed in U.S.patent application Ser. No. 09/941,148, entitled “Personalized RemoteControl,” which is based upon U.S. Provisional Patent Application Ser.No. 60/227,916, entitled “Personalized Remote Control,” referred to andincorporated above.

Further advantages of the various embodiments disclosed include theability to change embedded ads on a per-scene basis, and the ability toimplement embedded ads automatically or by manual methods. In addition,because embedded ads do not have to be interactive and can exist as partof the video signal. Televisions not enabled for interactivity are stillable to receive the embedded ads. Thus, the application of embedded adsmay not be restricted to video signals, but embedded ads may be appliedto any type of signal and any type of medium.

The various embodiments disclosed provide a system and method thatallows post-production visual alterations in the form of bid-foradvertisements to be added to a video stream. In this manner, broadcastand basic cable channels may provide bid-for embedded advertising tolocal companies in a simple and inexpensive manner. Visual alterations,or embedded ads, may be superimposed into the video stream manually orautomatically. Embedded ads are an effective way to advertise, andembedded ads may be superimposed into the video stream on ahouse-by-house basis. Embedded ads may be personalized and localized.Embedded ads may be applied to any form of viewing material, within anytype of signal or medium. Embedded ads may appear unobtrusively orobviously within the video content, and embedded ads may be eitherinteractive or non-interactive. Embedded ads may change on a per-scenebasis.

In addition, preferences and demographic information that controls thevarious types of ads that are displayed to users can be provided toadvertisers prior to bidding on the embedded advertising. Demographicand preference information that controls the displayed advertising isdisclosed in U.S. patent application Ser. No. 10/688,067, filed Oct. 16,2003, entitled “Demographic/Preference Sniffer,” which is based uponU.S. Provisional Patent Application Ser. No. 60/419,871, filed Oct. 18,2002, entitled “Demographic/Preference Sniffer,” which is specificallyincorporated herein by reference for all that it discloses and teaches.Further, upgraded advertising, such as disclosed in U.S. patentapplication Ser. No. 10/688,075, filed Oct. 17, 2003, entitled “iChooseVideo Advertising,” which is based upon U.S. Provisional ApplicationSer. No. 60/420,110, filed Oct. 18, 2002, entitled “iChoose VideoAdvertising,” which is specifically incorporated herein by reference forall that it discloses and teaches. Information regarding demographicsand preferences that control the display of information can be used tofurther aid the TV advertisers in making bids on embedded advertising.

FIG. 26 with a flow diagram that illustrates the process of auctioningembedded advertising. At step 2602, the broadcaster generates embeddedadvertising opportunities in the manner disclosed above. At step 2604,the broadcaster provides a list of embedded advertising opportunities toadvertisers and advertising specifications including required adinformation necessary to generate the embedded ad and other informationsuch as the expected duration of the ad, a description of the size ofthe ad, viewers who may be viewing the ad, targeted audiences,appearance of the ad and bidding information. There are targetedaudiences, as indicated above, that are based upon either demographicinformation or preference information of viewers. This information canbe provided to the advertiser prior to bidding. The broadcaster thenaccepts bids until the cutoff time at step 2606. At step 2608, thebroadcaster selects the winning bidder. At step 2610, the broadcasterinforms the bidders of the winning bidder and may provide additionalopportunities at that point in time for placing additional embedded adsto both the winning bidder and the losing bidders. At step 2614, thebroadcaster obtains the required ad information from the winning bidderprior to the ad deadline. The broadcaster then generates the embedded adat step 2616. At step 2618, the embedded ad is inserted into thebroadcast in the manner indicated above.

FIG. 27 is a flow diagram illustrating the steps 2700 that discloses theprocess performed by an advertiser in bidding on embedded advertising.At step 2702, the advertiser receives a list of embedded advertisingopportunities from the broadcaster. This list may be created, asdisclosed above, based upon attributes of the business of theadvertiser. For example, an advertiser may be in the business of sellingappliances, in which case, the advertiser may have listed “appliances”as an attribute. If there is an opportunity to place an embedded ad uponan appliance such as a refrigerator in a scene, the attribute“appliance” will be attached to the embedded advertising opportunity. Ofcourse, there may be many other examples of attributes which may be verybroad. As indicated above, the designation of an attribute of “reruns”which can be further limited to a type of programming, or a particularprogram, such as “situation comedy” or “Seinfeld” can be used as aseries of attributes so that an advertiser can obtain relevant embeddedadvertising information.

Referring again to FIG. 27, at step 2706, the advertiser reviews simplevideos, specifications for embedded ads, required ad information andbidding information. In this case, a simple video may be provided alongwith the embedded advertising opportunity which shows an ad embedded ina video clip so that the advertiser can have an idea as to the way inwhich the advertiser's ad may appear when the program is broadcast. Ofcourse, this process applies to prerecorded programming. However, withlive programming, such as a hockey game, video clips of previous gamesshowing embedded ads on the ice and on the boards can be provided as anexample of the manner in which an embedded ad may appear. Again, theadvertiser can determine the manner in which the advertiser's ad mayappear and may influence the amount that the advertiser may wish to bidon such ads.

At step 2708 of FIG. 27, the advertiser then determines if a bid will besubmitted. If not, the process proceeds to step 2710, and the advertiserreviews other embedded advertising opportunities. If the advertiserdetermines it wants to submit a bid, the process proceeds to step 2712to formulate a bidding strategy. Various bidding strategies are known inthe art of bidding and can be used by the advertiser. For example, anadvertiser may wish to submit a very high bid initially in an attempt todrive out other bidders. Of course, the advertiser may determine amaximum amount that the advertiser is willing to pay for the ad prior tosubmitting a first bid. In that regard, if affinity data is providedregarding the success of other embedded ads of the advertiser or thepopularity of the show, those factors may influence the maximum bidamount of the advertiser. At step 2714, the advertiser then submits abid. At step 2716, it is then determined whether the bid that has beensubmitted by the advertiser has won the auction. If it has not, theprocess proceeds to step 2722 where the advertiser may wish to reviewits bidding strategy. At step 2724, the advertiser then determineswhether it wishes to submit another bid. If not, the process proceeds tostep 2710 to review other advertising opportunities. If the advertiserdoes wish to submit another bid, the process returns to step 2714 tosubmit a bid. If the advertiser wins the bid at step 2716, the processproceeds to step 2718 where the advertiser provides the required adinformation to the TV broadcaster. The process then ends at step 2720.

FIG. 28 discloses the process 2800 of modifying bidding strategy basedupon affinity data. Affinity data is data that indicates viewers'responses to a program, an ad or any other information that isdisplayed. Affinity data can be collected by any desired methodincluding surveys, Nielsen rating data, or in accordance with theprocess described in U.S. patent application Ser. No. 10/046,618,entitled “Collection of Affinity Data from Television, Video or SimilarTransmissions,” filed Oct. 26, 2001, by Steven O. Markel, et al.

Referring again to FIG. 28, at step 2802, the advertiser determines thatit has won an advertising bid. At step 2804, the broadcaster generatesan enhanced ad. At step 2806, the enhanced ad is broadcast to viewers.At step 2808, affinity data is collected in accordance with any desiredmethod such as those described above. The affinity data is then gatheredby the advertiser and analyzed at step 2810. At step 2812, theadvertiser may wish to generate a new bidding strategy based upon theaffinity data. For example, if the advertiser is bidding on embedded adsfor the Super Bowl and the game is a very close game in the fourthquarter, the advertiser may wish to bid greater amounts. Of course, thereverse is true, also. For example, if an advertiser is bidding in theSuper Bowl and the score is very lopsided in the fourth quarter, theadvertiser may wish to bid much lower amounts or withdraw a previous bidand pay a penalty fee.

Affinity data can also be used to determine the success of a particularad. For example, affinity data can be collected with regard to theresponse of viewers to an embedded ad that has been displayed previouslyin a program. For example, affinity data can rate a viewer's response toan ad which provides valuable information to the advertiser as towhether the advertiser wishes to bid higher or lower on subsequent adsduring an event such as a football game or hockey game.

Of course, the processes disclosed above are not limited to embedded adsonly. For example, broadcasters may wish to designate certainadvertising time slots as ads which can be bid upon by advertisers. Inthis manner, a broadcaster may wish to maximize its revenue by havingsome time slots available for bidding while maintaining standardpre-purchased advertising time slots so that a guaranteed amount ofincome is provided. In addition, these time slots may have particularrelevance to what is occurring during the program. For example, a timeslot may be designated as a “bid-for” ad after a penalty is assessed ora play is being reviewed in a sporting event. An advertiser may wish topurchase that time slot and run a humorous advertisement showing areferee with thick glasses who has a hard time seeing. In addition, theaffinity data information can be very valuable for these types ofadvertisements to show viewer acceptance or rejection of theadvertisements. By collecting this data, advertisers can determine ifthey would like to run these ads on a more frequent basis and bid higherfor those particular time slots.

FIG. 29 discloses a bidding process 2900 that may be used for liveevents when the broadcaster is willing to extend the auction until atime that is very close to the time in which the embedded ad will bedisplayed. At step 2902, an advertiser receives the criteria foradvertising opportunities for the live event in the various ways asdisclosed above. At step 2904, the advertiser provides the requiredinformation for the embedded ads to the broadcaster. At step 2906, thebroadcaster generates the embedded ads for each advertiser that hassubmitted the required information and stores these embedded ads.Optionally, the broadcaster can provide a sample copy of the embedded adto advertisers. At step 2908, the broadcaster sets the end of auctiontime such that the end of auction time is close to the time for displayof the embedded ad. In this manner, bidding can occur during the processof the live event. For example, bidding may occur during actual play ofa game, such as the Stanley Cup finals, or the Super Bowl. The course ofevents in the game can cause the bids to increase or decrease during thegame as explained above. At step 2910, the bidders submit bids basedupon the progress of the live event. At step 2912, a broadcasterretrieves the embedded ad for the winning bidder. At step 2914, thebroadcaster displays the embedded ad.

FIG. 30 is a flow diagram illustrating the process 3000 of analyzingadvertising opportunities for live events. At step 3002, the broadcastercan review various camera angles and various views of a live event or asimilar event to attempt to determine embedded advertisingopportunities. For example, a broadcaster can view the camera angles orprevious video from a hockey game and determine what embeddedadvertising opportunities exist on the ice and on the boards of the icerink. As indicated above, even though markings may appear on the ice oron the boards, those markings can be removed using either markuplanguage overlay pages or video combination techniques, as disclosedabove. Hence, at step 3004, the potential locations for embedding adsare determined by the broadcaster. Advertisers may also assist thebroadcaster in that evaluation. At step 3006, the broadcaster collectsall of the required ad information from interested advertisers. At step3008, the various embedded ads (e.g., objects) are generated forinsertion in the live program. At step 3010, the broadcaster can thenprovide a sample display of the embedded ads such as an inserted object.For example, during a broadcast of a live football game, an insertedobject may comprise an airplane carrying a banner or a banner that isinserted into the crowd in the stands. Other embedded objects, asdisclosed above, can comprise ads that are placed on the playersthemselves, such as tattoos or labels. At step 3012, the broadcasterobtains suggestions from the advertiser on modifications and additionalembedded advertising opportunities.

FIG. 31 discloses the process 3100 for billing advertisers based upondisplay time of embedded ads. At step 3102, the broadcaster embeds thead in the programming in the manner described above. At step 3104, theembedding software counts the frames in which the full object (e.g., theembedded ad) is displayed on the screen. At step 3106, the total countof frames for each embedded object is provided to the billing software.At step 3108, the billing software generates a total advertising costfor the embedded ad based upon the total count of frames in which theembedded ad appears. Of course, a cap may exist that limits theadvertiser's exposure.

The present invention therefore provides various ways of allowingbroadcasters to auction both embedded ads and normal time slot ads.Auctioning of ads in this fashion allows broadcasters to potentiallyincrease advertising revenue for both prerecorded and live events.Advertisers, on the other hand, also benefit from the bidding process.For example, smaller companies may be able to advertise during largeevents such as the Super Bowl if the event has a smaller interestbecause of a lopsided score. Advertising can constitute targetedadvertising based upon preference and demographic data. In addition,affinity data, that evaluates the effect of the ad, can be used byadvertisers to maximize revenues and determine a bidding strategy.Embedded ads can be generated through markup language overlay pages orthrough combined video techniques. Various strategies for setting upembedded ads by broadcasters can be used including red zone advertisingfor football games, changing all of the embedded ads to a singleadvertiser when the home team scores and providing time slots that arecontent related to a particular advertisement. Attributes of an embeddedadvertising opportunity can be used to select potential advertisers,which also allow potential advertisers to only review embedded ads thatare of interest to the advertiser. Video combination techniques andmarkup language overlay pages can be used to erase existing ads in ascene and insert other ads or insert other objects into the scene. Forexample, if a table exists in a scene, a Coke can or beer can beactually inserted into the scene and an existing item such as an ashtraycan be removed. Similarly, an object such as a car can be erased and adifferent car can-be inserted in a scene to provide the embeddedadvertising opportunity.

Various advertising opportunities may be provided based upon theoccurrence of an event in the program, such as the scoring of a goal ora touchdown. In that case, a trigger may be inserted in the program thattriggers the insertion of one or more embedded ads in the program. Inthis example, a trigger may be embedded in the video program whichcauses the highest bidder who has bid upon the go-ahead goal on thehockey game or the go-ahead score in a football game, etc. to have allthe embedded ads changed to that particular advertiser's embedded ad.Various criteria can be set up such as scores during the fourth quarterof a football game, scores during the last few minutes of a hockey orbasketball game, etc. If advertisers wish to drop out of the biddingprocess, a backup fee for backing out of the process can be charged tothe advertiser. Advertisers can also bid upon putting embedded ad incertain camera angles. Hence, various methods are proposed forauctioning both embedded advertising and time slot advertising whichprovide many advantages to both broadcasters and to advertisers.

Referring now to FIG. 32a , a further embodiment of an implementationfor providing interactive advertisement is shown. The embodiment of FIG.32a is an enhancement of the FIG. 7 embodiment by providing access tosecondary content. The secondary content may be associated with thevideo being broadcast or the interactive advertisement, but is not aprimary product being advertised. For example in the embodimentillustrated in FIG. 32a , the embedded interactive advertisement isdirected to Paul's Pizzeria. However, during the display of theinteractive advertisement, a song may be playing. In this example, thesecondary content may be the song, a ringtone based on the song, a musicvideo for the song, and so forth, while the primary product is pizzafrom Paul's Pizzeria.

The user may activate an enhanced content activator associated with theinteractive advertisement to access further information including thesecondary content. Activation of the enhanced content activator mayoccur in several ways, including toggling through a series ofinteractive advertisements using control buttons on some form of ahand-held control device, such as a remote control, activating a cursorusing a wired or wireless mouse, or by other means including but not byway of limitation, light beams, such as IR beams that can be used aspointers to point at and select the interactive advertisement (e.g.,click on the Paul's Pizzeria logo).

It is noted that while example embodiments are discussed with respect toan interactive advertisement being embedded into a video broadcast,example embodiments may be directed to a non-embedded advertisement(e.g., video commercial during a commercial break from a show) that maybe activated to provide further content. For example, the non-embeddedadvertisement may be for a primary product of a soft drink, and stars aparticular singer. A song by the singer may be playing during thenon-embedded, interactive advertisement. Upon activation of theinteractive advertisement, the user may be provided, for example,secondary content triggers to obtain the secondary content (e.g., adownload of the song playing during the advertisement). In oneembodiment, the secondary content may be a physical product (e.g., aCD).

In FIG. 32a , secondary content triggers 3200 may be presented in anunused portion of a scene in a broadcast window 3202 to avoidinterruption of the current video presentation. If there is no unusedportion, then the secondary content triggers 3200 may be presented in aportion of the scene that has very little action. In an alternativeembodiment, the secondary content triggers 3200 may be provided along aborder, above, below, or to a side of the scene or the broadcast window3200, as will be discussed in more detail in the example of FIG. 32 b.

In example embodiments, the secondary content triggers 3200 allow theuser to access or download secondary content to their device or account.In one embodiment, the content may be downloaded to the account of theuser (e.g., in the cloud) and later accessed by the user. In the exampleof FIG. 32a , the secondary content triggers 3200 allows the user todownload the song being played, download a ringtone based on the song,or download a music video for the song. Alternative embodiments maycontemplate the use of any number of secondary content triggers 3200 andmay contemplate other forms of secondary content.

Upon selection of one of the secondary content triggers 3200, the usermay be asked to perform some actions prior to accessing the secondarycontent. For example, if payment is required, the user may be provided apayment interface. Alternatively or additionally, the user may berequired to authenticate themselves to the system (e.g., provide ausername/password combination or provide a PIN established for secondarycontent access). In some embodiments, the payment or authenticationinterface may be placed in the same area as the secondary contenttriggers 3200 (e.g., replacing the secondary content trigger display).

Referring now to FIG. 32b , an alternative embodiment of animplementation for providing interactive advertisement is shown. Theembodiment of FIG. 32b is based on the embodiment of FIG. 8. As shown,activation of interactive advertisement brings up the graphical userinterface (GUI) 802 on a side and bottom of the broadcast window 840 forproviding enhanced advertisement information. The GUI 802 presentsadditional information 804 (e.g., primary content) about the companyrelated to the primary product for which the advertisement has beenactivated, such as company profile 806, web address 808, coupons 810,local distributors/franchises 812, or a purchase trigger for theactivated advertisement (e.g., an “order pizza” website 814 or orderform). While the embodiment of FIG. 32b shows the GUI 802 as beingdisplayed to the right and bottom of the broadcast window 840, the GUI802 may be located on any number of sides and on any side of thebroadcast window 840.

As with the embodiment of FIG. 8, the enhanced advertisement informationalso includes access links that have nothing to do with the currentvideo presentation. That is, for example, various Internet(world-wide-web) activate links 816 and media interactive links (e.g.,Survivor vote link 818) may be provided. It is noted that in alternativeembodiments, these WWW activate links 816 and media interactive linksmay be optional or not included.

In the embodiment of FIG. 32b , secondary content triggers 3200 are alsoprovided in the enhanced advertisement information. As with theembodiment of FIG. 32a , these secondary content triggers 3200 allow theuser to access or download secondary content that is distinct from theprimary product of the advertisement to their device or account. In theexample of FIG. 32b , the secondary content triggers 3200 allows theuser to download the song being played, download a ringtone based on thesong, or download a music video for the song. Alternative embodimentsmay contemplate the use of any number of secondary content triggers 3200and may contemplate other forms of secondary content.

Referring now to FIG. 33, an example block diagram illustratingcomponents of a management system 3300 are shown. The management system3300 provides operations that enable the use of interactiveadvertisements, provisioning of enhanced advertisement information, andaccess to further content including secondary content. In exampleembodiments, the management system 3300 comprises an ad module 3302, anactivation module 3304, a data access module 3306, and a data storage3308, which may be communicatively coupled together. It is noted thatFIG. 33 provides one example of the management system 3300, andalternative embodiments may comprise more, fewer, or other components toenable the operations described herein. In one embodiment, themanagement system 3300 may be located on the server 1808, the centralcomputer 1810, the head-end, the set-top box, a transmission site, aretransmission site, or any other networked device. Alternatively,various components of the management system 3300 may be distributedacross and located at one or more of the server 1808, the centralcomputer 1810, the head-end, the set-top box, or any other networkeddevice.

The ad module 3302 provides interactive advertisements to a user device.In some embodiments, the ad module 3302 may identify advertisementopportunities in media or video programming (e.g., a television show,movie, sporting event). In some embodiments, the ad module 3302incorporates pattern recognition software and techniques to identifyavailable advertising space as discussed above. The pattern recognitionsoftware may communicate with the attribute database 1818 or an addatabase (not shown) to determine attributes to search for in the media.As previously discussed, the attributes may be actual object attributesor “blank space” attributes. The pattern recognition software may searchthe media (e.g. a video stream) for these actual objects or blankspaces. For example, a pizza box may be identified in the middle of ascene. The attributes associated with this object (e.g., pizza box) mayinclude object type (e.g., box or pizza box), dimensions of the object,location of the object, scene type, program type (e.g., comedy movie,sporting event, sitcom), and so forth. In some embodiments, a produceror distributor of the program or media may manually provide some ofthese attributes.

Additionally, the ad module 3302 may generate placement and contour datathat may include coordinates of the recognized object. As previouslydisclosed, placement and contour data may be generated by use ofcomputer recognition techniques, physical manipulation, or othertechniques.

The ad module 3302 further manages data stored for the advertisements.Initially, an advertiser or another entity (e.g., secondary contentprovider) may register with the management system 3300 via, for example,the ad module 3302. During or subsequent to registering with themanagement system 3300, the advertiser or another entity may provideinformation associated with their business (e.g., local franchises, homepage, address), their advertisement (e.g., coupons), or informationdirected to secondary content via the ad module 3302. The informationdirected to the secondary content may include, for example, links todownload secondary content, order forms for purchasing the secondarycontent, and metadata associated with the secondary content. Theinformation may be stored in the data storage 3308 by the ad module3302. In one embodiment, the data storage 3312 may be, or include, thedatabase 900.

The ad module 3302 may also search an advertiser database (e.g., datastorage 3312) for advertisements to embed into the advertisementopportunity. For example, advertisers that have one or more matchingattributes to those determined for the advertisement opportunity may beoffered the advertisement opportunity. The advertisement opportunity maybe offered to the advertisers via an auction or a fixed price mechanism.Once purchased by an advertiser, the ad module 3302 may embed anadvertisement from the advertiser into the advertisement opportunity.While the example embodiment describes the ad module 3302 as performingoperations directed to generating placement and contour data, managingdata stored for advertisers and secondary content providers, determiningadvertisers to present advertisement opportunities, and embeddingadvertisements, alternative embodiments may contemplate distributing oneor more of these operations between multiple modules.

The activation module 3304 receives an activation indication from theactivation of the enhanced content activator and provides enhancedadvertisement information in response. Once the advertisement isactivated and the activation indication received, the activation module3304 accesses the data storage 3312 for the enhanced advertisementinformation, and provisions the enhanced advertisement information fordisplay in, for example, an enhanced content window or border area.

The enhanced advertisement information may comprise triggers (e.g.,selectable links) to access further content and additional informationabout the company related to the primary product for which theadvertisement has been activated (e.g., company profile 806, web address808, coupons 810, local distributors/franchises 812, or a purchasetrigger for the activated advertisement such as an “order pizza” website814 or order form). The enhanced advertisement information may alsoinclude various Internet activate triggers (e.g., links 816) such asmedia interactive triggers (e.g., Survivor vote link 818, current eventslinks 824), and the secondary content triggers 3200.

The data access module 3306 provides the further content based on aselection of a selectable trigger provided by the activation module3304. In example embodiments, the data access module 3306 monitors forthe selection of the selectable trigger, and provides access to thefurther content based on the selection. In example embodiments, theaccess may comprise accessing an Internet resource (e.g., primaryadvertiser homepage, secondary content provider website), accessing thedata storage 3308 to retrieve the further content (e.g., stored digitalcontent for download), or a combination of both. In some cases, thefurther content may require payment or further authentication in orderto be accessed. The data access module 3306 may provide an interfacethat receives the payment or authentication information and provides itto the proper module or system for processing.

The data storage 3308 comprises a plurality of databases that are usedby the modules of the management system 3300. These databases mayinclude an advertisement database, advertiser database, and a secondarydata database. The advertisement database stores data regardingadvertisement opportunities. The data may include attributes, assignedvalues, identification of winning advertiser, and a copy of anadvertisement of the winning advertiser for each of the advertisementopportunities. The advertiser database stores data associated with theadvertiser. The secondary data database stores data for secondarycontent. With respect to the embodiment of FIG. 32b , for example, thesecondary data database may store a digital copy of the song or music,the ringtone for the song, or a digital copy of the video for the song.While the present embodiment provides a system that stores the secondarycontent at the management system 3300, alternative embodiments may storethe secondary content elsewhere within a network.

FIG. 34 is a flow diagram illustrating an example high-level method 3400for providing further content based on an interactive advertisement. Inoperation 3402, an advertisement opportunity is detected. In exampleembodiments, the ad module 3302 may identify advertisement opportunitiesin media (e.g., a television show, movie, sporting event). In someembodiments, the ad module 3302 incorporates pattern recognitionsoftware and techniques to identify available advertising space orgenerates placement and contour data that may include coordinates of therecognized object.

In operation 3404, one or more advertisers for the advertisementopportunity are determined. In example embodiments, the ad module 3302may search an advertiser database (e.g., data storage 3312) foradvertisements to embed into the advertisement opportunity. For example,advertisers that have one or more matching attributes to those indicatedby the advertisement opportunity may be offered the advertisementopportunity. The advertisement opportunity may be offered to theadvertisers via an auction or a fixed price mechanism.

Once purchased by an advertiser, the ad module 3302 may embed ormultiplex an interactive advertisement from the advertiser into theadvertisement opportunity in operation 3406. For example, theinteractive advertisement may be embedded into, or overlaid over, therecognized object of the advertisement opportunity using the placementand contour data for the recognized object. The ad module 3302 may thencause the display of the interactive advertisement and video content.For example, the interactive advertisement along with the video contentor media (sometimes referred to as a video program) may be provided to adevice of a user. The device of the user then displays the video programhaving the embedded interactive advertisement. It is noted that in someembodiments, the interactive advertisement is not embedded into videocontent or media, but instead, is a stand-alone advertisement orcommercial that appears between segments of video content. In theseembodiments, operation 3406 is not needed.

In operation 3408, an activation indication, based on activation of anenhanced content activator, is received that activates provisioning ofenhanced advertisement information. In example embodiments, theactivation module 3304 receives the activation indication from a userdevice that is displaying the interactive advertisement. In analternative embodiment, the activation module 3304 may receive theactivation indication from a selection device such as a remote control.

In response, the activation module 3304 causes the enhancedadvertisement information to be displayed in operation 3410. In oneembodiment, the activation module 3304 accesses a database (e.g., thedata storage 3312) for the enhanced advertisement information andprovisions the enhanced advertisement information to a user device fordisplay in, for example, an enhanced content window or border area(e.g., by multiplexing the enhanced advertisement information with thevideo program at the user device). In another embodiment, the enhancedadvertisement information may be multiplexed with the video program by acomponent of the management system 3330 (e.g., the data access module3306) and provided to the user device. The enhanced advertisementinformation may comprise triggers (e.g., selectable Internet activatelinks, media interactive links, and the secondary content triggers) toaccess further content and additional information about the companyrelated to the primary product for which the advertisement has beenactivated.

In operation 3412, the data access module 3306 monitors for a selectionof a trigger from the enhanced advertisement information. For example,the data access module 3306 may receive the selection of the triggerfrom the user device or a remote control device. Alternatively or inaddition, a remote control receiver circuit in a user device may monitorfor the selection.

In response to the selection, access to further content associated withthe selection is provided to the user device in operation 3414. Inexample embodiments, the data access module 3306 provides access to thefurther content based on the selection. The access may compriseproviding access to an Internet resource (e.g., primary advertiserhomepage, secondary content provider website), accessing the datastorage 3308 to retrieve the further content (e.g., stored digitalcontent for download), or a combination of both. In some cases, thefurther content may require payment or further authentication in orderto be accessed. The data access module 3306 may provide an interfacethat receives payment or authentication information and provides it tothe proper module or system for processing.

FIG. 35 is a flow diagram of a more detailed method (operation 3414) forproviding access to the further content based on activation of theinteractive advertisement. The operations of the method may be performedby the data access module 3306 or a processor configured by the dataaccess module 3306. In operation 3502, a determination is made as towhether the selection received in operation 3412 is directed torequesting additional (primary) advertiser information that is directlyrelated to the primary product being advertised. For example, if theadvertisement is for “Paul's Pizzeria,” then the primary product ispizza from Paul's Pizzeria and the additional advertiser information mayinclude location information, history, an order form, or a website forPaul's Pizzeria.

If the selection is directed to requesting additional advertiserinformation, then in operation 3504, the data access module 3306 willprovide access to the additional advertiser information. The additionaladvertiser information may include, for example, information about theadvertiser or product (e.g., company profile 806), a trigger for theadvertiser's website (e.g., web address 808), coupons or a link forcoupons for the advertiser (e.g., coupons 810), local information (e.g.,local distributors or franchises 812 information), or a direct purchasetrigger (e.g., order pizza link 814). The additional advertisementinformation may be accessed from, or provide access to, the network(e.g., the Internet) or be retrieved from a coupled data storage (e.g.,data storage 3308).

If the selection is not a request for additional advertiser information,a determination is made in operation 3506 whether the selection isdirected to a request for secondary content (e.g., content that isassociated with the primary product being advertised, but not theprimary product itself). If the selection is not a request for secondarycontent, then access to a remaining category or categories ofinformation is provided in operation 3508. The remaining categories ofinformation may include triggers or links granting access to resourceson the Internet (e.g., links 816). These triggers may include mediainteractive triggers (e.g., Survivor vote link 818 and current eventslinks 824). Alternatively, further content for the remaining categoriesmay be retrieved from a coupled data storage (e.g., data storage 3308).

However, if the selection is a request for secondary content, thenaccess to the secondary content is provided in operation 3510. Thesecondary content may be associated with the video being broadcast orthe embedded interactive advertisement, but is not directly related tothe primary product being advertised. For example the secondary contentmay be a song, a ringtone based on the song, a music video for the songthat is playing during the advertisement for a primary product that isnot the song. As such in operation 3510, a download of the song,ringtone, or music video may be accessed and provided to the userdevice.

In some cases, the secondary content may require payment or furtherauthentication prior to providing the secondary content. The data accessmodule 3306 may provide a user interface for receiving the payment orauthentication information. The payment or authentication information isthen provided to the proper module or system for processing.

Referring now to FIG. 36, an action flow diagram for providing additioncontent based on an interactive advertisement is shown. The variouscomponents involved with the process may include a control device 3602of a user or viewer that is communicatively coupled to a user device3604 that is located at a viewer location. The control device 3602 maybe, for example, a remote control, a mouse, keyboard, or any otherdevice for receiving user inputs. The user device 3604 may be embodiedwithin or coupled to, for example, a set-top box, interactivetelevision, or a computer and comprises a display screen. The userdevice 3604 may, in turn, be communicatively coupled to a managementsystem 3606 (e.g., a management system 3300) in accordance with oneexample embodiment. It is noted that in some embodiments, components ofthe management system 3606 may be embodied within, or is a part of, theuser device 3604. In other embodiments, components of the managementsystem 3606 may be located at the head-end, a transmission site, aretransmission site, a server system, or any other device networked tothe user device 3604. It is further noted that in some embodiments, thecontrol device 3602 may be a part of the user device 3604.

In some embodiments, the management system 3606 embeds an advertisementinto media (e.g., video program). As such, the management system 3606(e.g., an ad module 3302 of the management system 3606) may detect anadvertisement opportunity in the media or video program (e.g., atelevision show, movie, sporting event). In some cases, one or moreadvertisers for the advertisement opportunity are determined and theadvertisement opportunity may be offered to the advertisers via anauction or a fixed price mechanism. Subsequently, the management system3606 may embed or multiplex an interactive advertisement from theadvertiser into the advertisement opportunity in accordance with someembodiments. In alternative embodiments, the interactive advertisementmay be a standalone advertisement (e.g., not embedded into media).

The interactive advertisement along with the video program or media, ifany, is then transmitted the user device 3604, which causes the displayof the interactive advertisement on the user device 3604. The userdevice 3604 then may display, or cause display, of the media includingthe interactive advertisement on a display screen.

Using the control device 3602, the viewer may activate the advertisementin the media by activating an enhanced content activator. For example, auser may select, using the control device 3620, a displayed image of aproduct that is associated with the interactive advertisement toactivate the advertisement. In another example, the user may indicate onthe control device 3602 that any advertisement currently being viewedvia the user device 3604 be activated. The activation indication is sentto the management system 3606. In some embodiments, the activationindication may be sent directly to the management system 3606 by thecontrol device 3602, while in other embodiments, the activationindication may be initially sent to the user device 3604 and/orsubsequently forwarded to the management system 3606.

The management system 3606 receives the activation indication andprovisions the enhanced information (e.g., enhanced advertisementinformation) to cause the display of the enhanced information with thevideo program. In embodiments where the management system 3606 does notmaintain or store the enhanced information, the enhanced information maybe retrieved from a networked system or system (e.g., from theInternet).

Alternatively, if the management system 3606 maintains the enhancedinformation, the activation module 3304 may access a database (e.g., thedata storage 3312) for the enhanced advertisement information andprovide the enhanced advertisement information for display in, forexample, an enhanced content window or border area at the user device3604. The enhanced advertisement information may comprise triggers(e.g., selectable Internet activate links, media interactive links, andthe secondary content triggers) to access further content and additionalinformation about the company related to the primary product for whichthe advertisement has been activated.

In some embodiments, the enhanced information may be multiplexed withthe video program at the management system 3606 prior to transmission tothe user device 3604. Alternatively, the enhanced information may besent to the user device 3604 and a component of the user device 3604 maycombine, multiplex, or otherwise insert the enhanced information intothe video program. The communicating of the enhanced information may bevia the Internet or a broadcast system.

Using the control device 3602, a selection of a trigger from theenhanced advertisement information is made. The selected trigger is thensent to the management system 3606, either directly or via the userdevice 3604. In example embodiments, the data access module 3306 of themanagement system 3606 monitors for the selection of the trigger. Inother embodiments, a remote control receiver circuit of the user device3604 may monitor for the selection of the trigger and provide theselection information to the management system 3606 if the managementsystem 3606 is not embodied within the user device 3604.

In response to the selection, further content, or access to furthercontent, associated with the selection is provided to the user device3604 by the management system 3606. In example embodiments, the dataaccess module 3306 provides access to the further content based on theselection. The access may comprise providing access to an Internetresource (e.g., primary advertiser homepage, secondary content providerwebsite), accessing the data storage 3308 to retrieve the furthercontent (e.g., stored digital content for download), or a combination ofboth. The user device 3604 displays the further content to the viewer.

Modules, Components, and Logic

Additionally, certain embodiments described herein may be implemented aslogic or a number of modules, engines, components, or mechanisms. Amodule, engine, logic, component, or mechanism (collectively referred toas a “module”) may be a tangible unit capable of performing certainoperations and configured or arranged in a certain manner. In certainexample embodiments, one or more computer systems (e.g., a standalone,client, or server computer system) or one or more components of acomputer system (e.g., a processor or a group of processors) may beconfigured by software (e.g., an application or application portion) orfirmware (note that software and firmware can generally be usedinterchangeably herein as is known by a skilled artisan) as a modulethat operates to perform certain operations described herein.

In various embodiments, a module may be implemented mechanically orelectronically. For example, a module may comprise dedicated circuitryor logic that is permanently configured (e.g., within a special-purposeprocessor, application specific integrated circuit (ASIC), or array) toperform certain operations. A module may also comprise programmablelogic or circuitry (e.g., as encompassed within a general-purposeprocessor or other programmable processor) that is temporarilyconfigured by software or firmware to perform certain operations asdescribed herein. It will be appreciated that a decision to implement amodule mechanically, in dedicated and permanently configured circuitry,or in temporarily configured circuitry (e.g., configured by software)may be driven by, for example, cost, time, energy-usage, and packagesize considerations.

Accordingly, the term “module” should be understood to encompass atangible entity, be that an entity that is physically constructed,permanently configured (e.g., hardwired), or temporarily configured(e.g., programmed) to operate in a certain manner or to perform certainoperations described herein. Considering embodiments in which modules orcomponents are temporarily configured (e.g., programmed), each of themodules or components need not be configured or instantiated at any oneinstance in time. For example, where the modules or components comprisea general-purpose processor configured using software, thegeneral-purpose processor may be configured as respective differentmodules at different times. Software may accordingly configure theprocessor to constitute a particular module at one instance of time andto constitute a different module at a different instance of time. Theprocessor may be temporary configured or permanently configured toperform relevant operations. Whether temporarily configured orpermanently configured, such processors may constituteprocessor-implemented modules that operate to perform one or moreoperations or functions.

Modules can provide information to, and receive information from, othermodules. Accordingly, the described modules may be regarded as beingcommunicatively coupled. Where multiples of such modules existcontemporaneously, communications may be achieved through signaltransmission (e.g., over appropriate circuits and buses) that connectthe modules. In embodiments in which multiple modules are configured orinstantiated at different times, communications between such modules maybe achieved, for example, through the storage and retrieval ofinformation in memory structures to which the multiple modules haveaccess. For example, one module may perform an operation and store theoutput of that operation in a memory device to which it iscommunicatively coupled. A further module may then, at a later time,access the memory device to retrieve and process the stored output.Modules may also initiate communications with input or output devicesand can operate on a resource (e.g., a collection of information).

Example Machine Architecture and Machine-Readable Medium

With reference to FIG. 37, an example embodiment extends to a machine inthe example form of a computer system 3700 within which instructions forcausing the machine to perform any one or more of the methodologiesdiscussed herein may be executed. In alternative example embodiments,the machine operates as a standalone device or may be connected (e.g.,networked) to other machines. In a networked deployment, the machine mayoperate in the capacity of a server or a client machine in server-clientnetwork environment, or as a peer machine in a peer-to-peer (ordistributed) network environment. The machine may be a personal computer(PC), a tablet PC, a set-top box (STB), a Personal Digital Assistant(PDA), a cellular telephone, a web appliance, a network router, a switchor bridge, or any machine capable of executing instructions (sequentialor otherwise) that specify actions to be taken by that machine. Further,while only a single machine is illustrated, the term “machine” shallalso be taken to include any collection of machines that individually orjointly execute a set (or multiple sets) of instructions to perform anyone or more of the methodologies discussed herein.

The example computer system 3700 may include a processor 3702 (e.g., acentral processing unit (CPU), a graphics processing unit (GPU) orboth), a main memory 3704 and a static memory 3706, which communicatewith each other via a bus 3708. The computer system 3700 may furtherinclude a video display unit 3710 (e.g., a liquid crystal display (LCD)or a cathode ray tube (CRT)). In example embodiments, the computersystem 3700 also includes one or more of an alpha-numeric input device3712 (e.g., a keyboard), a user interface (UI) navigation device orcursor control device 3714 (e.g., a mouse), a disk drive unit 3716, asignal generation device 3718 (e.g., a speaker), and a network interfacedevice 3720.

Machine-Readable Storage Medium

The disk drive unit 3716 includes a machine-readable storage medium 3722on which is stored one or more sets of instructions 3724 and datastructures (e.g., software instructions) embodying or used by any one ormore of the methodologies or functions described herein. Theinstructions 3724 may also reside, completely or at least partially,within the main memory 3704 or within the processor 3702 duringexecution thereof by the computer system 3700, with the main memory 3704and the processor 3702 also constituting machine-readable media.

While the machine-readable storage medium 3722 is shown in an exampleembodiment to be a single medium, the term “machine-readable storagemedium” may include a single medium or multiple media (e.g., acentralized or distributed database, or associated caches and servers)that store the one or more instructions. The term “machine-readablemedium” shall also be taken to include any tangible medium that iscapable of storing, encoding, or carrying instructions for execution bythe machine and that cause the machine to perform any one or more of themethodologies of embodiments of the present invention, or that iscapable of storing, encoding, or carrying data structures used by orassociated with such instructions. The term “machine-readable storagemedium” shall accordingly be taken to include, but not be limited to,solid-state memories and optical and magnetic media. Specific examplesof machine-readable storage media include non-volatile memory, includingby way of example semiconductor memory devices (e.g., ErasableProgrammable Read-Only Memory (EPROM), Electrically ErasableProgrammable Read-Only Memory (EEPROM), and flash memory devices);magnetic disks such as internal hard disks and removable disks;magneto-optical disks; and CD-ROM and DVD-ROM disks.

Transmission Medium

The instructions 3724 may further be transmitted or received over acommunications network 3726 using a transmission medium via the networkinterface device 3720 and utilizing any one of a number of well-knowntransfer protocols (e.g., HTTP). Examples of communication networksinclude a local area network (LAN), a wide area network (WAN), theInternet, mobile telephone networks, POTS networks, and wireless datanetworks (e.g., WiFi and WiMax networks). The term “transmission medium”shall be taken to include any intangible medium that is capable ofstoring, encoding, or carrying instructions for execution by themachine, and includes digital or analog communications signals or otherintangible medium to facilitate communication of such software.

Although an overview of the inventive subject matter has been describedwith reference to specific example embodiments, various modificationsand changes may be made to these embodiments without departing from thebroader spirit and scope of embodiments of the present invention. Suchembodiments of the inventive subject matter may be referred to herein,individually or collectively, by the term “invention” merely forconvenience and without intending to voluntarily limit the scope of thisapplication to any single invention or inventive concept if more thanone is, in fact, disclosed.

The embodiments illustrated herein are described in sufficient detail toenable those skilled in the art to practice the teachings disclosed.Other embodiments may be used and derived therefrom, such thatstructural and logical substitutions and changes may be made withoutdeparting from the scope of this disclosure. The Detailed Description,therefore, is not to be taken in a limiting sense, and the scope ofvarious embodiments is defined only by the appended claims, along withthe full range of equivalents to which such claims are entitled.

As used herein, the term “or” may be construed in either an inclusive orexclusive sense. Additionally, although various example embodimentsdiscussed below focus on a insertion of localized advertisement datainto a national advertisement, the embodiments are given merely forclarity in disclosure. Thus, any insertion of localized advertisementdata into any type of media may employ various embodiments of the systemand method described herein and may be considered as being within ascope of example embodiments. Each of a variety of example embodimentsis discussed in detail below.

Moreover, plural instances may be provided for resources, operations, orstructures described herein as a single instance. Additionally,boundaries between various resources, operations, modules, engines, anddata stores are somewhat arbitrary, and particular operations areillustrated in a context of specific illustrative configurations. Otherallocations of functionality are envisioned and may fall within a scopeof various embodiments of the present invention. In general, structuresand functionality presented as separate resources in the exampleconfigurations may be implemented as a combined structure or resource.Similarly, structures and functionality presented as a single resourcemay be implemented as separate resources. These and other variations,modifications, additions, and improvements fall within a scope ofembodiments of the present invention as represented by the appendedclaims. The specification and drawings are, accordingly, to be regardedin an illustrative rather than a restrictive sense.

What is claimed is:
 1. A method comprising: causing a video program andan interactive advertisement to be displayed to a user on a displayscreen, the interactive advertisement including an enhanced contentactivator that causes display of enhanced advertisement information thatincludes a plurality of triggers to access further information;detecting a first input from the user that indicates an activation ofthe enhanced content activator of the interactive advertisement; inresponse to the detecting of the first input that indicates theactivation of the enhanced content activator by the user, provisioningenhanced advertisement information that includes the plurality oftriggers to access further information; causing the enhancedadvertisement information that includes the plurality of triggers to bedisplayed in combination with the video program and the interactiveadvertisement; receiving a second input from the user indicating aselection by the user of one trigger of the plurality of triggersincluded in the enhanced advertisement information displayed incombination with the video program and the interactive advertisement;and in response to the receiving the second input from the userindicating the selection of the one trigger, providing access to furthercontent corresponding to the selected one trigger.
 2. The method ofclaim 1, wherein the causing the video program and the interactiveadvertisement to be displayed to the user on the display screen isperformed on a user device.
 3. The method of claim 2, wherein the userdevice comprises a selection from the group consisting of a set-top box,a computer, a laptop, a smartphone, and a mobile device.
 4. The methodof claim 1, wherein the causing the video program and the interactiveadvertisement to be displayed to the user on the display screencomprises communicating the video program from a server system to a userdevice.
 5. The method of claim 1, wherein the provisioning the enhancedadvertisement information is performed at one or more servers.
 6. Themethod of claim 1, further comprising: in response to the detecting theactivation, accessing a data store to retrieve the enhancedadvertisement information associated with the enhanced contentactivator; and communicating the enhanced advertisement information tothe client system.
 7. The method of claim 1, wherein the causing theenhanced advertisement information to be displayed in combination withthe video program comprises multiplexing the enhanced advertisementinformation with the video program.
 8. The method of claim 1, whereinthe causing the enhanced advertisement information to be displayed incombination with the video program comprises transmitting an enhancedvideo content signal that includes the video program and the enhancedadvertisement information.
 9. The method of claim 8, further comprisingcommunicating the enhanced video content signal via Internet.
 10. Themethod of claim 8, further comprising communicating the enhanced videocontent signal via a broadcast system.
 11. The method of claim 1,wherein the receiving the second input indicating the selection by theuser is performed by a remote control receiver circuit in a user device.12. The method of claim 1, wherein the further content comprisessecondary content that is associated with the interactive advertisementbut not directly related to a primary product being advertised in theinteractive advertisement.
 13. The method of claim 1, wherein theinteractive advertisement is embedded into a portion of a scene with thevideo program.
 14. The method of claim 1, wherein the enhancedadvertisement information is provided in an unused or low activityportion of the video program.
 15. The method of claim 1, wherein theenhanced advertisement information associated with interactiveadvertisement is provided in an appended portion bordering at least oneside of a scene of the video program.
 16. The method of claim 1, whereinthe video program is a commercial that is the interactive advertisement.17. The method of claim 1, further comprising embedding the interactiveadvertisement into a portion of the video program using placement andcontour data derived by a pattern recognition process.
 18. The method ofclaim 1, further comprising identifying an advertisement opportunity bydetecting an object in the video program where the interactiveadvertisement is to be placed, the detecting comprises using a patternrecognition process to detect the object and to generate placement andcontour data for the detected object.
 19. The method of claim 1, whereinthe providing access to the further content comprises a selection fromthe group consisting of providing access to digital content fordownload, providing access to information from the Internet, andproviding an order interface to purchase a product.
 20. Amachine-readable storage medium in communication with at least oneprocessor, the non-transitory machine-readable medium storinginstructions which, when executed by the at least one processor of amachine, causes the machine to perform operations comprising: causing avideo program and an interactive advertisement to be displayed to a useron a display screen, the interactive advertisement including an enhancedcontent activator that causes display of enhanced advertisementinformation that includes a plurality of triggers to access furtherinformation; detecting a first input from the user that indicates anactivation of the enhanced content activator of the interactiveadvertisement; in response to the detecting of the first input thatindicates the activation of the enhanced content activator by the user,provisioning enhanced advertisement information that includes theplurality of triggers to access further information; causing theenhanced advertisement information that includes the plurality oftriggers to be displayed in combination with the video program and theinteractive advertisement; receiving a second input from the userindicating a selection by the user of one trigger of the plurality oftriggers included in the enhanced advertisement information displayed incombination with the video program and the interactive advertisement;and in response to the receiving second input from the user indicatingthe selection of the one trigger, providing access to further contentcorresponding to the selected one trigger.